Sep 10 04:39:12.345170 ip-10-0-136-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 04:39:12.345182 ip-10-0-136-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 04:39:12.345191 ip-10-0-136-230 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 04:39:12.345377 ip-10-0-136-230 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 04:39:22.394433 ip-10-0-136-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 10 04:39:22.394448 ip-10-0-136-230 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4eae02dd25164fcfbe935d2fe4c4c9ca -- Sep 10 04:41:13.797933 ip-10-0-136-230 systemd[1]: Starting Kubernetes Kubelet... Sep 10 04:41:14.153226 ip-10-0-136-230 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 04:41:14.153226 ip-10-0-136-230 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 04:41:14.153226 ip-10-0-136-230 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 04:41:14.153226 ip-10-0-136-230 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 04:41:14.153226 ip-10-0-136-230 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 04:41:14.154514 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.154427 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 04:41:14.158917 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158898 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 04:41:14.158917 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158915 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 04:41:14.158917 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158918 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 04:41:14.158917 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158921 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158924 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158930 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158933 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158937 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158940 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158943 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158945 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158948 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158951 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158953 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158956 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158959 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158962 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158964 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158967 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158970 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158973 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158976 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158978 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 04:41:14.159074 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158981 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158983 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158986 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158989 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158991 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158994 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.158996 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159006 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159009 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159011 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159014 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159016 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159019 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159022 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159025 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159028 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159030 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159033 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159036 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159038 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 04:41:14.159550 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159041 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159043 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159045 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159048 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159051 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159053 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159055 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159058 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159061 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159063 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159066 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159069 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159072 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159075 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159078 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159080 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159083 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159086 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159090 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 04:41:14.160058 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159094 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159097 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159100 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159103 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159105 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159108 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159111 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159114 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159117 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159119 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159122 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159124 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159127 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159129 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159131 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159134 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159137 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159140 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159143 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159145 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 04:41:14.160557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159148 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159151 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159155 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159158 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159161 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159164 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159167 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159169 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159172 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159174 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159177 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159180 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159183 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159185 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159188 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159191 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159193 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159885 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159891 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159896 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 04:41:14.161063 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159899 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159903 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159906 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159908 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159911 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159913 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159916 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159919 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159922 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159924 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159927 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159930 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159933 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159937 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159942 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159945 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159949 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159952 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 04:41:14.161540 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159956 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159958 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159961 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159964 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159967 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159970 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159973 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159976 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159978 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159981 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159983 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159986 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159988 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159992 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159995 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.159997 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160000 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160003 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160005 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 04:41:14.162021 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160008 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160011 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160013 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160016 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160019 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160021 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160024 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160027 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160029 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160033 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160036 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160038 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160041 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160043 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160046 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160049 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160051 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160054 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160056 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 04:41:14.162486 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160059 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160062 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160065 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160068 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160071 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160073 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160076 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160079 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160082 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160085 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160087 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160090 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160093 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160095 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160098 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160100 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160103 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160105 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160108 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160111 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 04:41:14.163036 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160114 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160117 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160120 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160122 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160128 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160131 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160134 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160136 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160139 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160142 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160144 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160147 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160149 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160152 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160155 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160158 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160160 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160165 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160168 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160171 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 04:41:14.163537 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160520 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160525 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160529 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160531 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160535 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160537 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160540 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160542 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160545 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160547 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160550 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160553 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160555 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160558 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160561 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160563 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160566 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160569 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160572 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160574 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 04:41:14.164043 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160577 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160579 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160582 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160584 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160587 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160589 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160592 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160595 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160597 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160599 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160618 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160622 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160624 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160627 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160630 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160635 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160638 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160642 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160645 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 04:41:14.164519 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160648 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160651 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160654 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160657 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160660 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160663 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160666 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160668 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160671 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160673 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160676 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160679 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160682 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160684 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160687 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160689 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160692 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160694 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160697 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160699 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 04:41:14.165001 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160702 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160704 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160707 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160709 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160713 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160715 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160718 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160720 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160725 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160727 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160730 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160732 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160735 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160737 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160740 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160743 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160745 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160747 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160750 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160753 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 04:41:14.165524 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160755 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160757 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160762 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160765 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160768 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160770 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160774 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160777 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160779 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160782 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160785 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160787 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160790 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160792 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160795 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160797 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160800 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160803 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160805 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 04:41:14.166033 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.160808 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160884 2565 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160919 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160924 2565 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160928 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160932 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160935 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160939 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160943 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160947 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160950 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160953 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160957 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160960 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160963 2565 flags.go:64] FLAG: --cgroup-root="" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160967 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160970 2565 flags.go:64] FLAG: --client-ca-file="" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160974 2565 flags.go:64] FLAG: --cloud-provider="external" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160977 2565 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160982 2565 flags.go:64] FLAG: --cluster-domain="" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160984 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160988 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160991 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160994 2565 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 04:41:14.166505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.160998 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161001 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161004 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161007 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161010 2565 flags.go:64] FLAG: --contention-profiling="false" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161013 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161016 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161020 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161023 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161026 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161029 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161032 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161035 2565 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161038 2565 flags.go:64] FLAG: --enable-server="true" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161041 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161045 2565 flags.go:64] FLAG: --event-burst="100" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161048 2565 flags.go:64] FLAG: --event-qps="50" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161051 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161054 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161057 2565 flags.go:64] FLAG: --eviction-hard="" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161060 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161063 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161066 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161070 2565 flags.go:64] FLAG: --eviction-soft="" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161074 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 04:41:14.167109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161076 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161080 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161083 2565 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161086 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161088 2565 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161091 2565 flags.go:64] FLAG: --feature-gates="" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161095 2565 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161097 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161100 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161103 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161106 2565 flags.go:64] FLAG: --healthz-port="10248" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161109 2565 flags.go:64] FLAG: --help="false" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161114 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161117 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161120 2565 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161123 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161126 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161129 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161132 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161135 2565 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161138 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161141 2565 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161143 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161147 2565 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 04:41:14.168067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161150 2565 flags.go:64] FLAG: --kube-reserved="" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161153 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161155 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161158 2565 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161161 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161164 2565 flags.go:64] FLAG: --lock-file="" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161168 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161171 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161174 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161179 2565 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161182 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161185 2565 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161188 2565 flags.go:64] FLAG: --logging-format="text" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161191 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161194 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161197 2565 flags.go:64] FLAG: --manifest-url="" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161199 2565 flags.go:64] FLAG: --manifest-url-header="" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161203 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161206 2565 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161210 2565 flags.go:64] FLAG: --max-pods="110" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161214 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161217 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161220 2565 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161223 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161226 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 04:41:14.169126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161229 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161232 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161239 2565 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161242 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161245 2565 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161248 2565 flags.go:64] FLAG: --pod-cidr="" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161251 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161256 2565 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161259 2565 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161262 2565 flags.go:64] FLAG: --pods-per-core="0" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161265 2565 flags.go:64] FLAG: --port="10250" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161268 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161271 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e86d01f3684ad0d" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161276 2565 flags.go:64] FLAG: --qos-reserved="" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161279 2565 flags.go:64] FLAG: --read-only-port="10255" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161282 2565 flags.go:64] FLAG: --register-node="true" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161284 2565 flags.go:64] FLAG: --register-with-taints="" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161288 2565 flags.go:64] FLAG: --registry-burst="10" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161292 2565 flags.go:64] FLAG: --registry-qps="5" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161295 2565 flags.go:64] FLAG: --reserved-cpus="" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161298 2565 flags.go:64] FLAG: --reserved-memory="" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161301 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161304 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161307 2565 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161310 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 04:41:14.170054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161313 2565 flags.go:64] FLAG: --runonce="false" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161315 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161319 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161323 2565 flags.go:64] FLAG: --seccomp-default="false" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161326 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161329 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161332 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161335 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161338 2565 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161341 2565 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161344 2565 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161347 2565 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161349 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161352 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161356 2565 flags.go:64] FLAG: --system-cgroups="" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161358 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161363 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161366 2565 flags.go:64] FLAG: --tls-cert-file="" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161369 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161373 2565 flags.go:64] FLAG: --tls-min-version="" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161376 2565 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161379 2565 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161382 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161385 2565 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161388 2565 flags.go:64] FLAG: --v="2" Sep 10 04:41:14.170785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161392 2565 flags.go:64] FLAG: --version="false" Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161396 2565 flags.go:64] FLAG: --vmodule="" Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161400 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.161404 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161509 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161512 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161515 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161519 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161523 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161526 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161530 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161533 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161536 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161539 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161542 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161545 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161548 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161551 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161554 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161557 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 04:41:14.171897 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161560 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161562 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161565 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161568 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161571 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161574 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161576 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161581 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161584 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161586 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161589 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161592 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161594 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161597 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161613 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161616 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161619 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161622 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161624 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161627 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 04:41:14.172468 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161629 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161632 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161637 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161639 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161642 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161644 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161647 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161650 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161653 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161656 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161658 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161661 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161664 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161666 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161669 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161671 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161674 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161678 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161681 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161685 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 04:41:14.173272 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161688 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161691 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161694 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161697 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161700 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161702 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161705 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161708 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161711 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161714 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161716 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161719 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161722 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161724 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161728 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161731 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161733 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161736 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161739 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161741 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 04:41:14.174098 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161744 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161747 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161750 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161752 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161755 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161758 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161760 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161763 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161766 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161768 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161771 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161775 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161778 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161780 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161783 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161785 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161788 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161791 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161793 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161796 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 04:41:14.174741 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161800 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161804 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.161807 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.162285 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.169689 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.169710 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.170593 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.170623 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170711 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170720 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170725 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170730 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170735 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170739 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170744 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 04:41:14.175270 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170748 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170755 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170762 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170768 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170773 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170777 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170782 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170786 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170790 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170794 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170799 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170803 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170807 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170812 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170816 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170820 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170824 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170829 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170834 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 04:41:14.175789 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170838 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170843 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170847 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170852 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170856 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170861 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170865 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170869 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170873 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170877 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170881 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170886 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170890 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170894 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170899 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170903 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170907 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170911 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170915 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170919 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 04:41:14.176312 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170923 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170927 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170932 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170936 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170940 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170944 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170948 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170952 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170957 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170961 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170965 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170970 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170977 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170982 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170987 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170991 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.170997 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171002 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171007 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 04:41:14.176828 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171012 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171016 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171021 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171025 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171030 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171034 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171037 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171042 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171046 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171051 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171055 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171059 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171064 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171068 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171072 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171076 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171080 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171084 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171089 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171093 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 04:41:14.177298 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171098 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171102 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171109 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171114 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171120 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171125 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171129 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171134 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171139 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171143 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171149 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171153 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171157 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171161 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 04:41:14.178087 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.171170 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171346 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171355 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171359 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171364 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171368 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171373 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171377 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171381 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171385 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171389 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171394 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171398 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171402 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171406 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171411 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171415 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171420 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171424 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 04:41:14.178682 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171428 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171432 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171436 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171441 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171445 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171450 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171454 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171458 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171462 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171467 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171471 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171476 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171480 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171484 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171488 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171493 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171497 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171501 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171505 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171509 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 04:41:14.179156 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171514 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171517 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171521 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171525 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171530 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171534 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171538 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171542 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171546 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171550 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171555 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171559 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171563 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171567 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171571 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171575 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171580 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171584 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171588 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171592 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 04:41:14.179740 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171596 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171617 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171623 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171628 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171632 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171637 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171641 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171645 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171649 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171653 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171658 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171662 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171666 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171670 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171674 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171679 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171684 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171688 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171692 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 04:41:14.180456 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171698 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171704 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171709 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171714 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171718 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171723 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171727 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171733 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171739 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171743 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171747 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171751 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171755 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171759 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171764 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171768 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171772 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171777 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171781 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 04:41:14.181200 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171787 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171792 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:14.171797 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.171806 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.171950 2565 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.174438 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.175377 2565 server.go:1019] "Starting client certificate rotation" Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.175476 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.175513 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 04:41:14.181974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.178073 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 04:41:14.187106 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.187087 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 04:41:14.200322 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.200306 2565 log.go:25] "Validated CRI v1 runtime API" Sep 10 04:41:14.203673 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.203658 2565 log.go:25] "Validated CRI v1 image API" Sep 10 04:41:14.204520 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.204506 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 04:41:14.205308 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.205292 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 04:41:14.206636 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.206595 2565 fs.go:135] Filesystem UUIDs: map[2d6107ac-c578-471a-9817-0d594147aac5:/dev/nvme0n1p3 6fd2146c-af5e-4767-8d8e-e1e4f5de44a6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 10 04:41:14.206698 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.206636 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 04:41:14.212048 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.211938 2565 manager.go:216] Machine: {Timestamp:2026-09-10 04:41:14.210691959 +0000 UTC m=+0.306158686 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102757 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec265535ffdc71b63ce8a344c46d4a3d SystemUUID:ec265535-ffdc-71b6-3ce8-a344c46d4a3d BootID:4eae02dd-2516-4fcf-be93-5d2fe4c4c9ca Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8a:00:34:5b:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8a:00:34:5b:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:c8:e0:7a:3b:63 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 04:41:14.212048 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.212042 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 04:41:14.212174 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.212120 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 04:41:14.214646 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.214619 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 04:41:14.214807 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.214649 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 04:41:14.215307 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.215295 2565 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 04:41:14.215361 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.215313 2565 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 04:41:14.215361 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.215324 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 04:41:14.216010 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.215999 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 04:41:14.216782 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.216771 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 04:41:14.220260 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220247 2565 state_mem.go:36] "Initialized new in-memory state store" Sep 10 04:41:14.220384 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220373 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 04:41:14.220447 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220413 2565 kubelet.go:486] "Attempting to sync node with API server" Sep 10 04:41:14.220447 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220434 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 04:41:14.220535 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220456 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 04:41:14.220535 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220468 2565 kubelet.go:398] "Adding apiserver pod source" Sep 10 04:41:14.220535 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220481 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 04:41:14.220876 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220862 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 04:41:14.221006 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.220887 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 04:41:14.223004 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.222978 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 04:41:14.224911 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.224894 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 04:41:14.224962 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.224927 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 04:41:14.226306 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226293 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 04:41:14.226344 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226314 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 04:41:14.226344 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226321 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 04:41:14.226344 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226326 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 04:41:14.226344 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226335 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 04:41:14.226344 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226341 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 04:41:14.226475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226355 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 04:41:14.226475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226361 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 04:41:14.226475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226374 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 04:41:14.226475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226379 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 04:41:14.226475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226388 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 04:41:14.226475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.226393 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 04:41:14.227036 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.227027 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 04:41:14.227036 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.227036 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 04:41:14.229223 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.229180 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g2c7q" Sep 10 04:41:14.231004 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.230983 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 04:41:14.231089 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.230999 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 04:41:14.231089 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.231049 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 04:41:14.232631 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.232600 2565 server.go:1268] "Started kubelet" Sep 10 04:41:14.232770 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.232707 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 04:41:14.232770 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.232735 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 04:41:14.232901 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.232803 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 04:41:14.233500 ip-10-0-136-230 systemd[1]: Started Kubernetes Kubelet. Sep 10 04:41:14.233810 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.233788 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g2c7q" Sep 10 04:41:14.234189 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.234170 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 04:41:14.234615 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.234588 2565 server.go:310] "Adding debug handlers to kubelet server" Sep 10 04:41:14.237752 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.236913 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-230.ec2.internal.18d3dbe9ce614590 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-230.ec2.internal,UID:ip-10-0-136-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-230.ec2.internal,},FirstTimestamp:2026-09-10 04:41:14.232563088 +0000 UTC m=+0.328029816,LastTimestamp:2026-09-10 04:41:14.232563088 +0000 UTC m=+0.328029816,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-230.ec2.internal,}" Sep 10 04:41:14.238549 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.238518 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 04:41:14.239263 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.239251 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 04:41:14.239687 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.239674 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 04:41:14.239821 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.239798 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.240238 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.240220 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 04:41:14.240238 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.240240 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 04:41:14.240351 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.240222 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 04:41:14.240351 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.240335 2565 reconstruct.go:97] "Volume reconstruction finished" Sep 10 04:41:14.240351 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.240340 2565 reconciler.go:29] "Reconciler: start to sync state" Sep 10 04:41:14.241232 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.241216 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 04:41:14.241232 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.241232 2565 factory.go:55] Registering systemd factory Sep 10 04:41:14.241356 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.241240 2565 factory.go:223] Registration of the systemd container factory successfully Sep 10 04:41:14.241474 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.241458 2565 factory.go:153] Registering CRI-O factory Sep 10 04:41:14.241530 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.241476 2565 factory.go:223] Registration of the crio container factory successfully Sep 10 04:41:14.241530 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.241496 2565 factory.go:103] Registering Raw factory Sep 10 04:41:14.241530 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.241517 2565 manager.go:1196] Started watching for new ooms in manager Sep 10 04:41:14.242020 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.242006 2565 manager.go:349] Starting recovery of all containers Sep 10 04:41:14.250470 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.250360 2565 manager.go:354] Recovery completed Sep 10 04:41:14.251799 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.251776 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 04:41:14.254131 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.254107 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-230.ec2.internal\" not found" node="ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.256736 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.256722 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 04:41:14.258663 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.258627 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientMemory" Sep 10 04:41:14.258663 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.258649 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 04:41:14.258663 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.258659 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientPID" Sep 10 04:41:14.259302 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.259284 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 04:41:14.259302 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.259297 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 04:41:14.259410 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.259313 2565 state_mem.go:36] "Initialized new in-memory state store" Sep 10 04:41:14.259905 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.259891 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 04:41:14.260698 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.260680 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientMemory" Sep 10 04:41:14.260771 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.260711 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 04:41:14.260771 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.260726 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientPID" Sep 10 04:41:14.261400 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.261389 2565 policy_none.go:49] "None policy: Start" Sep 10 04:41:14.261437 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.261405 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 04:41:14.261437 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.261414 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 04:41:14.263443 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.263431 2565 policy_none.go:47] "Start" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.299669 2565 manager.go:337] "Starting Device Plugin manager" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.299704 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.299715 2565 server.go:87] "Starting device plugin registration server" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.300091 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.300102 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.300196 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.300292 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.300301 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.300993 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 04:41:14.324704 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.301046 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.377616 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.377575 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 04:41:14.378721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.378705 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 04:41:14.378792 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.378755 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 04:41:14.378792 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.378780 2565 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 04:41:14.378856 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.378823 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 04:41:14.380977 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.380960 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 04:41:14.400701 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.400677 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 04:41:14.402554 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.402539 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientMemory" Sep 10 04:41:14.402644 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.402567 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 04:41:14.402644 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.402587 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientPID" Sep 10 04:41:14.402644 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.402625 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.408205 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.408132 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.408205 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.408154 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-230.ec2.internal\": node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.417751 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.417735 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.479404 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.479372 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal"] Sep 10 04:41:14.479552 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.479458 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 04:41:14.480918 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.480903 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientMemory" Sep 10 04:41:14.480987 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.480931 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 04:41:14.480987 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.480941 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientPID" Sep 10 04:41:14.483155 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483143 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 04:41:14.483282 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483267 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.483316 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483301 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 04:41:14.483879 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483859 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientMemory" Sep 10 04:41:14.483879 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483879 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 04:41:14.484046 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483889 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientPID" Sep 10 04:41:14.484046 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483888 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientMemory" Sep 10 04:41:14.484046 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483934 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 04:41:14.484046 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.483952 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientPID" Sep 10 04:41:14.486025 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.486010 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.486089 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.486036 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 04:41:14.486687 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.486675 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientMemory" Sep 10 04:41:14.486738 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.486696 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 04:41:14.486738 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.486707 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeHasSufficientPID" Sep 10 04:41:14.507191 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.507173 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-230.ec2.internal\" not found" node="ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.511076 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.511059 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-230.ec2.internal\" not found" node="ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.517822 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.517806 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.618353 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.618323 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.641661 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.641633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5866f7e54ad2a0195ec26f63896ba833-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal\" (UID: \"5866f7e54ad2a0195ec26f63896ba833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.641734 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.641667 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5866f7e54ad2a0195ec26f63896ba833-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal\" (UID: \"5866f7e54ad2a0195ec26f63896ba833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.641734 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.641688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d97b5ec804f662bb91372d0961fb0075-config\") pod \"kube-apiserver-proxy-ip-10-0-136-230.ec2.internal\" (UID: \"d97b5ec804f662bb91372d0961fb0075\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.718921 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.718891 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.742400 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.742374 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d97b5ec804f662bb91372d0961fb0075-config\") pod \"kube-apiserver-proxy-ip-10-0-136-230.ec2.internal\" (UID: \"d97b5ec804f662bb91372d0961fb0075\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.742472 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.742414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5866f7e54ad2a0195ec26f63896ba833-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal\" (UID: \"5866f7e54ad2a0195ec26f63896ba833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.742472 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.742433 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5866f7e54ad2a0195ec26f63896ba833-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal\" (UID: \"5866f7e54ad2a0195ec26f63896ba833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.742537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.742480 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d97b5ec804f662bb91372d0961fb0075-config\") pod \"kube-apiserver-proxy-ip-10-0-136-230.ec2.internal\" (UID: \"d97b5ec804f662bb91372d0961fb0075\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.742537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.742483 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5866f7e54ad2a0195ec26f63896ba833-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal\" (UID: \"5866f7e54ad2a0195ec26f63896ba833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.742537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.742483 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5866f7e54ad2a0195ec26f63896ba833-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal\" (UID: \"5866f7e54ad2a0195ec26f63896ba833\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.809546 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.809514 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.814225 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:14.814210 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" Sep 10 04:41:14.819066 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.819045 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:14.919634 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:14.919586 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:15.020131 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:15.020100 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:15.051321 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.051299 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 04:41:15.122156 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:15.122124 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:15.175338 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.175312 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 04:41:15.175764 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.175451 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 04:41:15.175764 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.175484 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 04:41:15.175764 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.175484 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 04:41:15.222995 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:15.222973 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:15.238348 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.238317 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 04:36:15 +0000 UTC" deadline="2028-05-17 01:10:12.76216785 +0000 UTC" Sep 10 04:41:15.238348 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.238344 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14756h28m57.523826384s" Sep 10 04:41:15.323406 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:15.323373 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:15.422090 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:15.422050 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5866f7e54ad2a0195ec26f63896ba833.slice/crio-a7b8bac66ad8297483c22e54950a012f71177099a302b7da19791542368e7dd3 WatchSource:0}: Error finding container a7b8bac66ad8297483c22e54950a012f71177099a302b7da19791542368e7dd3: Status 404 returned error can't find the container with id a7b8bac66ad8297483c22e54950a012f71177099a302b7da19791542368e7dd3 Sep 10 04:41:15.423801 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:15.423780 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-230.ec2.internal\" not found" Sep 10 04:41:15.424129 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:15.424108 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd97b5ec804f662bb91372d0961fb0075.slice/crio-45f0bd9f06d183cfd5b36d042797d96fc9f27547cc59bd1dc699556ccad52c8c WatchSource:0}: Error finding container 45f0bd9f06d183cfd5b36d042797d96fc9f27547cc59bd1dc699556ccad52c8c: Status 404 returned error can't find the container with id 45f0bd9f06d183cfd5b36d042797d96fc9f27547cc59bd1dc699556ccad52c8c Sep 10 04:41:15.426923 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.426907 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 04:41:15.512711 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.512674 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 04:41:15.540206 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.540143 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" Sep 10 04:41:15.552014 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.551990 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 04:41:15.553066 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.553050 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" Sep 10 04:41:15.562044 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:15.562022 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 04:41:16.156927 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.156896 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 04:41:16.221414 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.221384 2565 apiserver.go:52] "Watching apiserver" Sep 10 04:41:16.237804 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.237596 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 04:41:16.238160 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.238131 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-jtx8l","kube-system/konnectivity-agent-vhws4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn","openshift-cluster-node-tuning-operator/tuned-rvr7s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-mrd8j","openshift-multus/network-metrics-daemon-44mnf","openshift-network-diagnostics/network-check-target-8pzhh","openshift-ovn-kubernetes/ovnkube-node-s9cp4","kube-system/global-pull-secret-syncer-f2xkr","kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal","openshift-dns/node-resolver-4n6vl","openshift-image-registry/node-ca-bllrf","openshift-multus/multus-hrctp"] Sep 10 04:41:16.239542 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.239510 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 04:41:16.240934 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.240915 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:16.241172 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.241137 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:16.243368 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.243347 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:16.243467 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.243410 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:16.245959 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.245510 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.248030 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.248009 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 04:41:16.248682 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.248264 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 04:41:16.248984 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.248331 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 04:41:16.249239 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.248392 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-97kqx\"" Sep 10 04:41:16.249601 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.248456 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.251117 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.250652 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.251375 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.251355 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 04:41:16.252044 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.252027 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 04:41:16.252224 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.252206 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rs2d9\"" Sep 10 04:41:16.252318 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.252069 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 04:41:16.252572 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.252547 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:16.252677 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.252587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:16.252677 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.252643 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w7qb\" (UniqueName: \"kubernetes.io/projected/54664d95-d774-4a37-a7f1-eb37c2efb5ea-kube-api-access-8w7qb\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:16.252979 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.252958 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 04:41:16.253201 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.253179 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jqlb8\"" Sep 10 04:41:16.253404 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.253385 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 04:41:16.253689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.253654 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 04:41:16.253841 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.253706 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.256675 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.256547 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 04:41:16.256851 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.256837 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-czx29\"" Sep 10 04:41:16.256929 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.256866 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 04:41:16.256929 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.256907 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 04:41:16.257036 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.256936 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 04:41:16.257036 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.256837 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 04:41:16.257143 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.257132 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.258876 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.258853 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.259554 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.259532 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 04:41:16.259554 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.259548 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 04:41:16.259779 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.259762 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pjh9c\"" Sep 10 04:41:16.259851 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.259767 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 04:41:16.259965 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.259920 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 04:41:16.261512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.260863 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 04:41:16.261512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.260913 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 04:41:16.261656 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.261523 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.262285 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.262267 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dlhwz\"" Sep 10 04:41:16.263120 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.262535 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 04:41:16.263120 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.262557 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 04:41:16.263120 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.262602 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 04:41:16.264156 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.263895 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 04:41:16.264707 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.264333 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 04:41:16.264707 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.264618 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.268713 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.268675 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.268822 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.268769 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dgqjw\"" Sep 10 04:41:16.269042 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.268986 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:16.270769 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.270729 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.271292 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.271269 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 04:41:16.272254 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.272023 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n5fjh\"" Sep 10 04:41:16.273949 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.273554 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 04:41:16.273949 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.273598 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 04:41:16.273949 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.273792 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-26dpz\"" Sep 10 04:41:16.277067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.276984 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hk45g" Sep 10 04:41:16.284891 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.284864 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hk45g" Sep 10 04:41:16.341890 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.341865 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353436 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-cni-bin\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353461 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-daemon-config\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353484 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-hosts-file\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353502 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-cni-netd\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-kubelet\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353533 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-log-socket\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353586 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-kubelet\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353639 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnw56\" (UniqueName: \"kubernetes.io/projected/6f8d8b07-d521-43ac-b9cb-3e753107f87c-kube-api-access-vnw56\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqsks\" (UniqueName: \"kubernetes.io/projected/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-kube-api-access-gqsks\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.353689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-var-lib-kubelet\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3-konnectivity-ca\") pod \"konnectivity-agent-vhws4\" (UID: \"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3\") " pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353737 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a308358a-d20b-40d7-b9bc-a95c79d0c32a-host\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysctl-d\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353779 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-lib-modules\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353794 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-system-cni-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353825 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-var-lib-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-etc-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-ovnkube-config\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-host\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353958 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-node-log\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a308358a-d20b-40d7-b9bc-a95c79d0c32a-serviceca\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.353999 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-cni-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354014 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-socket-dir-parent\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.354276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-netns\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354063 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-cni-multus\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgpdz\" (UniqueName: \"kubernetes.io/projected/c417b44d-13b8-4555-83df-7e5229f14823-kube-api-access-sgpdz\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-tuned\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-conf-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-multus-certs\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354167 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-registration-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-modprobe-d\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354232 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysctl-conf\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354252 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f8d8b07-d521-43ac-b9cb-3e753107f87c-cni-binary-copy\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354284 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-etc-selinux\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvgw8\" (UniqueName: \"kubernetes.io/projected/abbbc61a-acc8-43d5-9e41-7054f79a9156-kube-api-access-tvgw8\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-cnibin\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354347 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h85b2\" (UniqueName: \"kubernetes.io/projected/ac0af446-e525-4216-adfc-6188ecc148a5-kube-api-access-h85b2\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354365 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f065d09c-3151-4335-bf0d-05452ac3aa70-dbus\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.355070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354388 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mgjf\" (UniqueName: \"kubernetes.io/projected/c15d5817-f692-4b74-8f30-0174ed0dfba9-kube-api-access-4mgjf\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354420 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-slash\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354442 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-etc-kubernetes\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354485 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w7qb\" (UniqueName: \"kubernetes.io/projected/54664d95-d774-4a37-a7f1-eb37c2efb5ea-kube-api-access-8w7qb\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-system-cni-dir\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354724 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-run-netns\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354755 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-run\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354793 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4e431b1b-f695-46dc-862d-ed9f9590a5b7-iptables-alerter-script\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-hostroot\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354859 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-systemd\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-ovnkube-script-lib\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354917 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjp5s\" (UniqueName: \"kubernetes.io/projected/4e431b1b-f695-46dc-862d-ed9f9590a5b7-kube-api-access-gjp5s\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354962 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3-agent-certs\") pod \"konnectivity-agent-vhws4\" (UID: \"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3\") " pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.354987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-tmp-dir\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.355883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355012 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-sys-fs\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355064 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-systemd-units\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355089 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-cni-bin\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355129 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac0af446-e525-4216-adfc-6188ecc148a5-ovn-node-metrics-cert\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355152 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c15d5817-f692-4b74-8f30-0174ed0dfba9-tmp\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355180 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-k8s-cni-cncf-io\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-env-overrides\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355234 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2qnl\" (UniqueName: \"kubernetes.io/projected/a308358a-d20b-40d7-b9bc-a95c79d0c32a-kube-api-access-h2qnl\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f065d09c-3151-4335-bf0d-05452ac3aa70-kubelet-config\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysconfig\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355308 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-kubernetes\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355331 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-systemd\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e431b1b-f695-46dc-862d-ed9f9590a5b7-host-slash\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-socket-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355424 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:16.356729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355449 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.357420 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-sys\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.357420 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355496 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-cnibin\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.357420 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355528 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-device-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.357420 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355552 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.357420 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.355645 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-ovn\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.357420 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.355786 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:16.357420 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.355872 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs podName:54664d95-d774-4a37-a7f1-eb37c2efb5ea nodeName:}" failed. No retries permitted until 2026-09-10 04:41:16.855838482 +0000 UTC m=+2.951305211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs") pod "network-metrics-daemon-44mnf" (UID: "54664d95-d774-4a37-a7f1-eb37c2efb5ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:16.361191 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.361164 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 04:41:16.361831 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.361776 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 04:41:16.361831 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.361800 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 04:41:16.361831 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.361812 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6tgsn for pod openshift-network-diagnostics/network-check-target-8pzhh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:16.362105 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.361892 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn podName:94cbf17b-159d-43da-af57-639c03a0cabd nodeName:}" failed. No retries permitted until 2026-09-10 04:41:16.861871652 +0000 UTC m=+2.957338382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6tgsn" (UniqueName: "kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn") pod "network-check-target-8pzhh" (UID: "94cbf17b-159d-43da-af57-639c03a0cabd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:16.365905 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.365879 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w7qb\" (UniqueName: \"kubernetes.io/projected/54664d95-d774-4a37-a7f1-eb37c2efb5ea-kube-api-access-8w7qb\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:16.384997 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.384905 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" event={"ID":"d97b5ec804f662bb91372d0961fb0075","Type":"ContainerStarted","Data":"45f0bd9f06d183cfd5b36d042797d96fc9f27547cc59bd1dc699556ccad52c8c"} Sep 10 04:41:16.386394 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.386367 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" event={"ID":"5866f7e54ad2a0195ec26f63896ba833","Type":"ContainerStarted","Data":"a7b8bac66ad8297483c22e54950a012f71177099a302b7da19791542368e7dd3"} Sep 10 04:41:16.455958 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.455917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysctl-d\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.455958 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.455967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-lib-modules\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.455986 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-system-cni-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456004 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-var-lib-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456076 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-var-lib-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456135 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456139 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-system-cni-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456144 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysctl-d\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-etc-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456194 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-ovnkube-config\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456218 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-host\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.456228 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456227 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-lib-modules\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456244 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456272 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-node-log\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a308358a-d20b-40d7-b9bc-a95c79d0c32a-serviceca\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456317 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-cni-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456357 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-socket-dir-parent\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456389 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-netns\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-cni-multus\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgpdz\" (UniqueName: \"kubernetes.io/projected/c417b44d-13b8-4555-83df-7e5229f14823-kube-api-access-sgpdz\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456450 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456465 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-tuned\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456469 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-etc-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456488 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-conf-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456274 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-host\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456514 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-cni-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456515 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-multus-certs\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-node-log\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456563 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-multus-certs\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.456818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-registration-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-socket-dir-parent\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456642 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-cni-multus\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456658 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-modprobe-d\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456698 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysctl-conf\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f8d8b07-d521-43ac-b9cb-3e753107f87c-cni-binary-copy\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456751 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-etc-selinux\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456779 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvgw8\" (UniqueName: \"kubernetes.io/projected/abbbc61a-acc8-43d5-9e41-7054f79a9156-kube-api-access-tvgw8\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-cnibin\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456831 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h85b2\" (UniqueName: \"kubernetes.io/projected/ac0af446-e525-4216-adfc-6188ecc148a5-kube-api-access-h85b2\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456847 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-ovnkube-config\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456906 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-modprobe-d\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456920 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f065d09c-3151-4335-bf0d-05452ac3aa70-dbus\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456856 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f065d09c-3151-4335-bf0d-05452ac3aa70-dbus\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456984 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.456992 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a308358a-d20b-40d7-b9bc-a95c79d0c32a-serviceca\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.457794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457124 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysctl-conf\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457170 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-netns\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457220 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-etc-selinux\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457223 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-cnibin\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mgjf\" (UniqueName: \"kubernetes.io/projected/c15d5817-f692-4b74-8f30-0174ed0dfba9-kube-api-access-4mgjf\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457276 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-registration-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457293 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-slash\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457318 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-etc-kubernetes\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457346 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457352 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-slash\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-system-cni-dir\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457442 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-run-netns\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457466 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-run\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457490 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4e431b1b-f695-46dc-862d-ed9f9590a5b7-iptables-alerter-script\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457496 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f8d8b07-d521-43ac-b9cb-3e753107f87c-cni-binary-copy\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457515 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-hostroot\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.458658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-systemd\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457565 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-run-netns\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457575 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-run\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457581 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-ovnkube-script-lib\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457606 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-etc-kubernetes\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjp5s\" (UniqueName: \"kubernetes.io/projected/4e431b1b-f695-46dc-862d-ed9f9590a5b7-kube-api-access-gjp5s\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457647 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-systemd\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457651 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c417b44d-13b8-4555-83df-7e5229f14823-system-cni-dir\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457648 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3-agent-certs\") pod \"konnectivity-agent-vhws4\" (UID: \"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3\") " pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457700 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-tmp-dir\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457729 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-sys-fs\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-systemd-units\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457794 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-cni-bin\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac0af446-e525-4216-adfc-6188ecc148a5-ovn-node-metrics-cert\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c15d5817-f692-4b74-8f30-0174ed0dfba9-tmp\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-k8s-cni-cncf-io\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-env-overrides\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457320 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-conf-dir\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.459586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2qnl\" (UniqueName: \"kubernetes.io/projected/a308358a-d20b-40d7-b9bc-a95c79d0c32a-kube-api-access-h2qnl\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457946 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f065d09c-3151-4335-bf0d-05452ac3aa70-kubelet-config\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysconfig\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-kubernetes\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-systemd\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e431b1b-f695-46dc-862d-ed9f9590a5b7-host-slash\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-socket-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458135 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-sys\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-cnibin\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458193 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-device-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458220 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458232 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4e431b1b-f695-46dc-862d-ed9f9590a5b7-iptables-alerter-script\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458249 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-ovn\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458280 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458311 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-cni-bin\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458348 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-daemon-config\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.460587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458373 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-hosts-file\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458423 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-cni-netd\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458429 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-hosts-file\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-kubelet\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-log-socket\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458501 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-kubelet\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458537 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnw56\" (UniqueName: \"kubernetes.io/projected/6f8d8b07-d521-43ac-b9cb-3e753107f87c-kube-api-access-vnw56\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqsks\" (UniqueName: \"kubernetes.io/projected/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-kube-api-access-gqsks\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458645 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-var-lib-kubelet\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458670 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3-konnectivity-ca\") pod \"konnectivity-agent-vhws4\" (UID: \"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3\") " pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a308358a-d20b-40d7-b9bc-a95c79d0c32a-host\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458806 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a308358a-d20b-40d7-b9bc-a95c79d0c32a-host\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458870 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-kubelet\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.458855 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-ovnkube-script-lib\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459037 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-var-lib-kubelet\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459127 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e431b1b-f695-46dc-862d-ed9f9590a5b7-host-slash\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.461482 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459183 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-kubelet\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459209 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-tmp-dir\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459277 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-sys-fs\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459325 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-systemd-units\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459366 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-cni-netd\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459386 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459418 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-log-socket\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459460 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-cni-bin\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459511 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-run-k8s-cni-cncf-io\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459616 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3-konnectivity-ca\") pod \"konnectivity-agent-vhws4\" (UID: \"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3\") " pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459701 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-sysconfig\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459712 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f065d09c-3151-4335-bf0d-05452ac3aa70-kubelet-config\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459741 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-hostroot\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.457891 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c417b44d-13b8-4555-83df-7e5229f14823-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459780 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-kubernetes\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459814 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-systemd\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459836 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-device-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459907 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac0af446-e525-4216-adfc-6188ecc148a5-env-overrides\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.462419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459948 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/abbbc61a-acc8-43d5-9e41-7054f79a9156-socket-dir\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.459983 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c15d5817-f692-4b74-8f30-0174ed0dfba9-sys\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.460006 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-cnibin\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.460037 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.460078 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-openvswitch\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.460083 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.460089 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f8d8b07-d521-43ac-b9cb-3e753107f87c-host-var-lib-cni-bin\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.460140 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac0af446-e525-4216-adfc-6188ecc148a5-run-ovn\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.460146 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret podName:f065d09c-3151-4335-bf0d-05452ac3aa70 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:16.960126064 +0000 UTC m=+3.055592787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret") pod "global-pull-secret-syncer-f2xkr" (UID: "f065d09c-3151-4335-bf0d-05452ac3aa70") : object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.460888 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3-agent-certs\") pod \"konnectivity-agent-vhws4\" (UID: \"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3\") " pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.462564 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac0af446-e525-4216-adfc-6188ecc148a5-ovn-node-metrics-cert\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.462786 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c15d5817-f692-4b74-8f30-0174ed0dfba9-etc-tuned\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.463349 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.462996 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6f8d8b07-d521-43ac-b9cb-3e753107f87c-multus-daemon-config\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.464048 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.463906 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c15d5817-f692-4b74-8f30-0174ed0dfba9-tmp\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.466545 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.466517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgpdz\" (UniqueName: \"kubernetes.io/projected/c417b44d-13b8-4555-83df-7e5229f14823-kube-api-access-sgpdz\") pod \"multus-additional-cni-plugins-mrd8j\" (UID: \"c417b44d-13b8-4555-83df-7e5229f14823\") " pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.467267 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.467245 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvgw8\" (UniqueName: \"kubernetes.io/projected/abbbc61a-acc8-43d5-9e41-7054f79a9156-kube-api-access-tvgw8\") pod \"aws-ebs-csi-driver-node-r88pn\" (UID: \"abbbc61a-acc8-43d5-9e41-7054f79a9156\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.469088 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.469062 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnw56\" (UniqueName: \"kubernetes.io/projected/6f8d8b07-d521-43ac-b9cb-3e753107f87c-kube-api-access-vnw56\") pod \"multus-hrctp\" (UID: \"6f8d8b07-d521-43ac-b9cb-3e753107f87c\") " pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.469400 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.469376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqsks\" (UniqueName: \"kubernetes.io/projected/7d2881f6-4630-4769-90e6-ea5c9af3f4d8-kube-api-access-gqsks\") pod \"node-resolver-4n6vl\" (UID: \"7d2881f6-4630-4769-90e6-ea5c9af3f4d8\") " pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.469719 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.469676 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjp5s\" (UniqueName: \"kubernetes.io/projected/4e431b1b-f695-46dc-862d-ed9f9590a5b7-kube-api-access-gjp5s\") pod \"iptables-alerter-jtx8l\" (UID: \"4e431b1b-f695-46dc-862d-ed9f9590a5b7\") " pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.469922 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.469859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h85b2\" (UniqueName: \"kubernetes.io/projected/ac0af446-e525-4216-adfc-6188ecc148a5-kube-api-access-h85b2\") pod \"ovnkube-node-s9cp4\" (UID: \"ac0af446-e525-4216-adfc-6188ecc148a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.471229 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.471204 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2qnl\" (UniqueName: \"kubernetes.io/projected/a308358a-d20b-40d7-b9bc-a95c79d0c32a-kube-api-access-h2qnl\") pod \"node-ca-bllrf\" (UID: \"a308358a-d20b-40d7-b9bc-a95c79d0c32a\") " pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.471854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.471835 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mgjf\" (UniqueName: \"kubernetes.io/projected/c15d5817-f692-4b74-8f30-0174ed0dfba9-kube-api-access-4mgjf\") pod \"tuned-rvr7s\" (UID: \"c15d5817-f692-4b74-8f30-0174ed0dfba9\") " pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.562326 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.562295 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" Sep 10 04:41:16.574628 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.574597 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" Sep 10 04:41:16.586696 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.586673 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jtx8l" Sep 10 04:41:16.593682 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.593658 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" Sep 10 04:41:16.601496 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.601475 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:16.610746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.610729 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bllrf" Sep 10 04:41:16.617515 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.617493 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:16.624339 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.624316 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hrctp" Sep 10 04:41:16.632095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.632077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4n6vl" Sep 10 04:41:16.775889 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.775806 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 04:41:16.862412 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.862368 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:16.862412 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.862418 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:16.862643 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.862549 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:16.862643 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.862633 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 04:41:16.862762 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.862645 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs podName:54664d95-d774-4a37-a7f1-eb37c2efb5ea nodeName:}" failed. No retries permitted until 2026-09-10 04:41:17.862603726 +0000 UTC m=+3.958070460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs") pod "network-metrics-daemon-44mnf" (UID: "54664d95-d774-4a37-a7f1-eb37c2efb5ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:16.862762 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.862650 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 04:41:16.862762 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.862670 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6tgsn for pod openshift-network-diagnostics/network-check-target-8pzhh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:16.862762 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.862713 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn podName:94cbf17b-159d-43da-af57-639c03a0cabd nodeName:}" failed. No retries permitted until 2026-09-10 04:41:17.862698585 +0000 UTC m=+3.958165309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tgsn" (UniqueName: "kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn") pod "network-check-target-8pzhh" (UID: "94cbf17b-159d-43da-af57-639c03a0cabd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:16.921837 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.921787 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 04:41:16.963505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:16.963475 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:16.963708 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.963660 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:16.963783 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:16.963737 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret podName:f065d09c-3151-4335-bf0d-05452ac3aa70 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:17.963713534 +0000 UTC m=+4.059180253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret") pod "global-pull-secret-syncer-f2xkr" (UID: "f065d09c-3151-4335-bf0d-05452ac3aa70") : object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:17.287845 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:17.287785 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 04:36:17 +0000 UTC" deadline="2028-03-19 07:06:16.500517743 +0000 UTC" Sep 10 04:41:17.287845 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:17.287838 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13346h24m59.21268553s" Sep 10 04:41:17.379876 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:17.379847 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:17.380027 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.379955 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:17.419817 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.419580 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b82cf9f_b435_4bd4_bd01_bbba2dcdd6a3.slice/crio-eac8b02e12762116335f74aef8009f3088b2c7ccbebdb291f235d71e517ee9f8 WatchSource:0}: Error finding container eac8b02e12762116335f74aef8009f3088b2c7ccbebdb291f235d71e517ee9f8: Status 404 returned error can't find the container with id eac8b02e12762116335f74aef8009f3088b2c7ccbebdb291f235d71e517ee9f8 Sep 10 04:41:17.420749 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.420724 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc417b44d_13b8_4555_83df_7e5229f14823.slice/crio-d4a2b546b3b2bf06b084fb7a5da17c2427ddb2392586d402d70e8d587e4576a8 WatchSource:0}: Error finding container d4a2b546b3b2bf06b084fb7a5da17c2427ddb2392586d402d70e8d587e4576a8: Status 404 returned error can't find the container with id d4a2b546b3b2bf06b084fb7a5da17c2427ddb2392586d402d70e8d587e4576a8 Sep 10 04:41:17.421988 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.421950 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0af446_e525_4216_adfc_6188ecc148a5.slice/crio-3a503dd2592edbe5be334f68131fd9cae2f5ada6034e15eea66361f4abba2495 WatchSource:0}: Error finding container 3a503dd2592edbe5be334f68131fd9cae2f5ada6034e15eea66361f4abba2495: Status 404 returned error can't find the container with id 3a503dd2592edbe5be334f68131fd9cae2f5ada6034e15eea66361f4abba2495 Sep 10 04:41:17.425797 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.425771 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d2881f6_4630_4769_90e6_ea5c9af3f4d8.slice/crio-2d7523edd74694f8b2ba6557d113a8d46fcea75ad25e14eb8a167b2d4506d91e WatchSource:0}: Error finding container 2d7523edd74694f8b2ba6557d113a8d46fcea75ad25e14eb8a167b2d4506d91e: Status 404 returned error can't find the container with id 2d7523edd74694f8b2ba6557d113a8d46fcea75ad25e14eb8a167b2d4506d91e Sep 10 04:41:17.426945 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.426923 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f8d8b07_d521_43ac_b9cb_3e753107f87c.slice/crio-94429e80ffface094f9d58b368835883c6c336775226c399a9096a2948295bd4 WatchSource:0}: Error finding container 94429e80ffface094f9d58b368835883c6c336775226c399a9096a2948295bd4: Status 404 returned error can't find the container with id 94429e80ffface094f9d58b368835883c6c336775226c399a9096a2948295bd4 Sep 10 04:41:17.427779 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.427756 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15d5817_f692_4b74_8f30_0174ed0dfba9.slice/crio-f94ac5c6d757f25cfc38f3d1a620d0fa46ad797228b4a251d6b2b8b33b9a4338 WatchSource:0}: Error finding container f94ac5c6d757f25cfc38f3d1a620d0fa46ad797228b4a251d6b2b8b33b9a4338: Status 404 returned error can't find the container with id f94ac5c6d757f25cfc38f3d1a620d0fa46ad797228b4a251d6b2b8b33b9a4338 Sep 10 04:41:17.429174 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.428773 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda308358a_d20b_40d7_b9bc_a95c79d0c32a.slice/crio-b8af99254850705d9e7249ec09572a6a547601cd240c06a18a48417702a323a7 WatchSource:0}: Error finding container b8af99254850705d9e7249ec09572a6a547601cd240c06a18a48417702a323a7: Status 404 returned error can't find the container with id b8af99254850705d9e7249ec09572a6a547601cd240c06a18a48417702a323a7 Sep 10 04:41:17.431631 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:17.431549 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbbc61a_acc8_43d5_9e41_7054f79a9156.slice/crio-504dff20fa46dc54277efeeef2638856b659133eb14bc16f54b5aa2f5fb54070 WatchSource:0}: Error finding container 504dff20fa46dc54277efeeef2638856b659133eb14bc16f54b5aa2f5fb54070: Status 404 returned error can't find the container with id 504dff20fa46dc54277efeeef2638856b659133eb14bc16f54b5aa2f5fb54070 Sep 10 04:41:17.870377 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:17.870335 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:17.870561 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:17.870401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:17.870561 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.870499 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:17.870696 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.870573 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs podName:54664d95-d774-4a37-a7f1-eb37c2efb5ea nodeName:}" failed. No retries permitted until 2026-09-10 04:41:19.870549076 +0000 UTC m=+5.966015807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs") pod "network-metrics-daemon-44mnf" (UID: "54664d95-d774-4a37-a7f1-eb37c2efb5ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:17.870696 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.870572 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 04:41:17.870696 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.870605 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 04:41:17.870696 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.870639 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6tgsn for pod openshift-network-diagnostics/network-check-target-8pzhh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:17.870696 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.870678 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn podName:94cbf17b-159d-43da-af57-639c03a0cabd nodeName:}" failed. No retries permitted until 2026-09-10 04:41:19.870666112 +0000 UTC m=+5.966132832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tgsn" (UniqueName: "kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn") pod "network-check-target-8pzhh" (UID: "94cbf17b-159d-43da-af57-639c03a0cabd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:17.971470 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:17.971414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:17.971663 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.971600 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:17.971746 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:17.971682 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret podName:f065d09c-3151-4335-bf0d-05452ac3aa70 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:19.971662099 +0000 UTC m=+6.067128831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret") pod "global-pull-secret-syncer-f2xkr" (UID: "f065d09c-3151-4335-bf0d-05452ac3aa70") : object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:18.289032 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.288919 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 04:36:17 +0000 UTC" deadline="2028-03-29 15:43:19.468576718 +0000 UTC" Sep 10 04:41:18.289032 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.288980 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13595h2m1.179600844s" Sep 10 04:41:18.380359 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.379835 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:18.380359 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:18.379986 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:18.380359 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.380199 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:18.380359 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:18.380310 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:18.404441 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.404229 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" event={"ID":"d97b5ec804f662bb91372d0961fb0075","Type":"ContainerStarted","Data":"d34fb19fa742e538b7e9580aae29bbe7599ee1371226112c50671177eb1bc4bf"} Sep 10 04:41:18.411214 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.411146 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bllrf" event={"ID":"a308358a-d20b-40d7-b9bc-a95c79d0c32a","Type":"ContainerStarted","Data":"b8af99254850705d9e7249ec09572a6a547601cd240c06a18a48417702a323a7"} Sep 10 04:41:18.417989 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.417941 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" event={"ID":"c15d5817-f692-4b74-8f30-0174ed0dfba9","Type":"ContainerStarted","Data":"f94ac5c6d757f25cfc38f3d1a620d0fa46ad797228b4a251d6b2b8b33b9a4338"} Sep 10 04:41:18.424063 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.424034 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hrctp" event={"ID":"6f8d8b07-d521-43ac-b9cb-3e753107f87c","Type":"ContainerStarted","Data":"94429e80ffface094f9d58b368835883c6c336775226c399a9096a2948295bd4"} Sep 10 04:41:18.438293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.438263 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"3a503dd2592edbe5be334f68131fd9cae2f5ada6034e15eea66361f4abba2495"} Sep 10 04:41:18.443852 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.443825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerStarted","Data":"d4a2b546b3b2bf06b084fb7a5da17c2427ddb2392586d402d70e8d587e4576a8"} Sep 10 04:41:18.460386 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.460358 2565 generic.go:356] "Generic (PLEG): container finished" podID="5866f7e54ad2a0195ec26f63896ba833" containerID="16a6ffe1502cd2091f8c193dccbd0739feacb8a89ab93a074d2f41ee7a3731e2" exitCode=0 Sep 10 04:41:18.460559 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.460536 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" event={"ID":"5866f7e54ad2a0195ec26f63896ba833","Type":"ContainerDied","Data":"16a6ffe1502cd2091f8c193dccbd0739feacb8a89ab93a074d2f41ee7a3731e2"} Sep 10 04:41:18.484809 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.484752 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-230.ec2.internal" podStartSLOduration=2.484733114 podStartE2EDuration="2.484733114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:18.420158784 +0000 UTC m=+4.515625526" watchObservedRunningTime="2026-09-10 04:41:18.484733114 +0000 UTC m=+4.580199858" Sep 10 04:41:18.494400 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.494306 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" event={"ID":"abbbc61a-acc8-43d5-9e41-7054f79a9156","Type":"ContainerStarted","Data":"504dff20fa46dc54277efeeef2638856b659133eb14bc16f54b5aa2f5fb54070"} Sep 10 04:41:18.496486 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.496422 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jtx8l" event={"ID":"4e431b1b-f695-46dc-862d-ed9f9590a5b7","Type":"ContainerStarted","Data":"f9dab82f068b4e6119009bbad6bcac123160d24382d96e5aa1a01165cd01c8b1"} Sep 10 04:41:18.517698 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.517657 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4n6vl" event={"ID":"7d2881f6-4630-4769-90e6-ea5c9af3f4d8","Type":"ContainerStarted","Data":"2d7523edd74694f8b2ba6557d113a8d46fcea75ad25e14eb8a167b2d4506d91e"} Sep 10 04:41:18.541209 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:18.541091 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhws4" event={"ID":"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3","Type":"ContainerStarted","Data":"eac8b02e12762116335f74aef8009f3088b2c7ccbebdb291f235d71e517ee9f8"} Sep 10 04:41:19.379719 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:19.379140 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:19.379719 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:19.379288 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:19.584578 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:19.584495 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" event={"ID":"5866f7e54ad2a0195ec26f63896ba833","Type":"ContainerStarted","Data":"485150b1ee60bc68b5f546639272a1deb7596d3a21dc7af5fa614928127283bf"} Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:19.898944 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:19.899002 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:19.899143 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:19.899224 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs podName:54664d95-d774-4a37-a7f1-eb37c2efb5ea nodeName:}" failed. No retries permitted until 2026-09-10 04:41:23.899201415 +0000 UTC m=+9.994668149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs") pod "network-metrics-daemon-44mnf" (UID: "54664d95-d774-4a37-a7f1-eb37c2efb5ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:19.899229 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:19.899251 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:19.899267 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6tgsn for pod openshift-network-diagnostics/network-check-target-8pzhh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:19.899358 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:19.899326 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn podName:94cbf17b-159d-43da-af57-639c03a0cabd nodeName:}" failed. No retries permitted until 2026-09-10 04:41:23.899310271 +0000 UTC m=+9.994777005 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tgsn" (UniqueName: "kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn") pod "network-check-target-8pzhh" (UID: "94cbf17b-159d-43da-af57-639c03a0cabd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:20.000123 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:20.000078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:20.000301 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:20.000290 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:20.000385 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:20.000350 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret podName:f065d09c-3151-4335-bf0d-05452ac3aa70 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:24.000331808 +0000 UTC m=+10.095798532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret") pod "global-pull-secret-syncer-f2xkr" (UID: "f065d09c-3151-4335-bf0d-05452ac3aa70") : object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:20.383200 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:20.382476 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:20.383200 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:20.382611 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:20.383200 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:20.383039 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:20.383200 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:20.383138 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:21.380632 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:21.379945 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:21.380632 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:21.380154 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:22.379947 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:22.379818 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:22.379947 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:22.379820 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:22.380582 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:22.379922 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:22.380582 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:22.380045 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:23.379976 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:23.379887 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:23.380463 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:23.380048 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:23.938511 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:23.938470 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:23.938749 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:23.938536 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:23.938824 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:23.938755 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 04:41:23.938824 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:23.938780 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 04:41:23.938824 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:23.938794 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6tgsn for pod openshift-network-diagnostics/network-check-target-8pzhh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:23.938987 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:23.938860 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn podName:94cbf17b-159d-43da-af57-639c03a0cabd nodeName:}" failed. No retries permitted until 2026-09-10 04:41:31.938838562 +0000 UTC m=+18.034305300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tgsn" (UniqueName: "kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn") pod "network-check-target-8pzhh" (UID: "94cbf17b-159d-43da-af57-639c03a0cabd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:23.939280 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:23.939261 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:23.939362 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:23.939317 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs podName:54664d95-d774-4a37-a7f1-eb37c2efb5ea nodeName:}" failed. No retries permitted until 2026-09-10 04:41:31.939301497 +0000 UTC m=+18.034768240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs") pod "network-metrics-daemon-44mnf" (UID: "54664d95-d774-4a37-a7f1-eb37c2efb5ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:24.039397 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:24.039352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:24.039586 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:24.039533 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:24.039646 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:24.039601 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret podName:f065d09c-3151-4335-bf0d-05452ac3aa70 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:32.039580806 +0000 UTC m=+18.135047530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret") pod "global-pull-secret-syncer-f2xkr" (UID: "f065d09c-3151-4335-bf0d-05452ac3aa70") : object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:24.380262 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:24.380182 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:24.380656 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:24.380304 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:24.380760 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:24.380737 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:24.380871 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:24.380849 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:25.379554 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:25.379503 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:25.379761 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:25.379704 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:26.380738 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:26.380706 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:26.381170 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:26.380848 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:26.381341 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:26.381321 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:26.381429 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:26.381411 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:27.379537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:27.379227 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:27.379702 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:27.379677 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:28.379784 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:28.379752 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:28.380227 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:28.379754 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:28.380227 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:28.379863 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:28.380227 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:28.379901 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:29.379563 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:29.379530 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:29.379747 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:29.379682 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:30.379435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:30.379397 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:30.379902 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:30.379407 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:30.379902 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:30.379518 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:30.379902 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:30.379653 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:31.379431 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:31.379396 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:31.379690 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:31.379524 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:32.001614 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:32.001564 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:32.001771 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:32.001674 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:32.001771 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.001744 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 04:41:32.001771 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.001756 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:32.001771 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.001766 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 04:41:32.001939 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.001778 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6tgsn for pod openshift-network-diagnostics/network-check-target-8pzhh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:32.001939 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.001818 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs podName:54664d95-d774-4a37-a7f1-eb37c2efb5ea nodeName:}" failed. No retries permitted until 2026-09-10 04:41:48.001800179 +0000 UTC m=+34.097266909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs") pod "network-metrics-daemon-44mnf" (UID: "54664d95-d774-4a37-a7f1-eb37c2efb5ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:32.001939 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.001831 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn podName:94cbf17b-159d-43da-af57-639c03a0cabd nodeName:}" failed. No retries permitted until 2026-09-10 04:41:48.001825626 +0000 UTC m=+34.097292343 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tgsn" (UniqueName: "kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn") pod "network-check-target-8pzhh" (UID: "94cbf17b-159d-43da-af57-639c03a0cabd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:32.102849 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:32.102813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:32.103026 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.102963 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:32.103098 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.103032 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret podName:f065d09c-3151-4335-bf0d-05452ac3aa70 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:48.103014043 +0000 UTC m=+34.198480758 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret") pod "global-pull-secret-syncer-f2xkr" (UID: "f065d09c-3151-4335-bf0d-05452ac3aa70") : object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:32.379839 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:32.379772 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:32.380157 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.379869 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:32.380157 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:32.379933 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:32.380157 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:32.380037 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:33.378993 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:33.378959 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:33.379168 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:33.379086 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:34.380237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:34.380205 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:34.380683 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:34.380164 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:34.380683 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:34.380596 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:34.380808 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:34.380773 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:35.379199 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:35.379169 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:35.379374 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:35.379294 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:36.380085 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.379861 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:36.380891 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.379900 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:36.380891 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:36.380195 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:36.380891 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:36.380304 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:36.624871 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.624783 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bllrf" event={"ID":"a308358a-d20b-40d7-b9bc-a95c79d0c32a","Type":"ContainerStarted","Data":"0bfe205d5fc3cc81ac1292a2088fb4931c721af7fc3c5d5fc39b468319067cf1"} Sep 10 04:41:36.626425 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.626395 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" event={"ID":"c15d5817-f692-4b74-8f30-0174ed0dfba9","Type":"ContainerStarted","Data":"4607b66dccb50ac287eb45f0cebd01d60251a133166e5294233aeafab8f00164"} Sep 10 04:41:36.628102 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.628059 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hrctp" event={"ID":"6f8d8b07-d521-43ac-b9cb-3e753107f87c","Type":"ContainerStarted","Data":"ef82c3e901b0d01496e5801fd33dd4d8f0919ff06d0688bc218c5f61035d6f35"} Sep 10 04:41:36.631509 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.631489 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"6cd9e774092e607154678608f1f8fe66e90de9c86468c8cf7f06064f60b2c5dc"} Sep 10 04:41:36.631634 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.631549 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"6ee49622e4e91d6835c50df460613c387b475899937d5770ac2abd6b368cdf2c"} Sep 10 04:41:36.631634 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.631592 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"5f21607a3bb3ac7c383513280ce2bbd06768a71986a1a99a2186d87425fbb316"} Sep 10 04:41:36.631731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.631653 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"048efb0430e9d696ea3a4db9af077e3624253a6ece0b47a0ba482d4b90de8e62"} Sep 10 04:41:36.631731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.631684 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"6e8c2fbd7f5a4fc72adcf650e1ab0474f827073ea11c29a04f2549e4f36894e4"} Sep 10 04:41:36.631731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.631718 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"70dcfb379a114571a28f2e0099bb6023034c1557c909cafafacbf273a201408b"} Sep 10 04:41:36.633192 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.633169 2565 generic.go:356] "Generic (PLEG): container finished" podID="c417b44d-13b8-4555-83df-7e5229f14823" containerID="95a3c27a81c8aec54ec6b02da0441bc6128acc1e590fd5afb5430886fb5a466b" exitCode=0 Sep 10 04:41:36.633310 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.633245 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerDied","Data":"95a3c27a81c8aec54ec6b02da0441bc6128acc1e590fd5afb5430886fb5a466b"} Sep 10 04:41:36.634961 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.634936 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" event={"ID":"abbbc61a-acc8-43d5-9e41-7054f79a9156","Type":"ContainerStarted","Data":"646fbcb427bc68663c8cfeb3b3cfb512ff563ccd6ac2540f00f29544f060a7c4"} Sep 10 04:41:36.636652 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.636630 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4n6vl" event={"ID":"7d2881f6-4630-4769-90e6-ea5c9af3f4d8","Type":"ContainerStarted","Data":"1ac212facbc25d4a51af7c2cba8559eef702480b6014194e4e39f7cbcc3ffc46"} Sep 10 04:41:36.638269 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.638242 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhws4" event={"ID":"2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3","Type":"ContainerStarted","Data":"85db6d27c9187490f091feeed0a39ce95d3e7a3c21609b7b21650e3d1c435a67"} Sep 10 04:41:36.639052 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.639010 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-230.ec2.internal" podStartSLOduration=20.638993852 podStartE2EDuration="20.638993852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:19.601965764 +0000 UTC m=+5.697432503" watchObservedRunningTime="2026-09-10 04:41:36.638993852 +0000 UTC m=+22.734460591" Sep 10 04:41:36.653764 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.653722 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rvr7s" podStartSLOduration=3.329808955 podStartE2EDuration="21.653710425s" totalImagesPullingTime="18.32390147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:36.653527912 +0000 UTC m=+22.748994674" watchObservedRunningTime="2026-09-10 04:41:36.653710425 +0000 UTC m=+22.749177162" Sep 10 04:41:36.654252 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.654219 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bllrf" podStartSLOduration=3.332801 podStartE2EDuration="21.654209935s" totalImagesPullingTime="18.321408935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:36.638794479 +0000 UTC m=+22.734261217" watchObservedRunningTime="2026-09-10 04:41:36.654209935 +0000 UTC m=+22.749676672" Sep 10 04:41:36.667648 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.667591 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vhws4" podStartSLOduration=6.893074811 podStartE2EDuration="21.667576619s" totalImagesPullingTime="14.774501808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:36.667533453 +0000 UTC m=+22.763000189" watchObservedRunningTime="2026-09-10 04:41:36.667576619 +0000 UTC m=+22.763043356" Sep 10 04:41:36.682155 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.682122 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hrctp" podStartSLOduration=3.306903836 podStartE2EDuration="21.68211056s" totalImagesPullingTime="18.375206724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:36.682111673 +0000 UTC m=+22.777578412" watchObservedRunningTime="2026-09-10 04:41:36.68211056 +0000 UTC m=+22.777577294" Sep 10 04:41:36.695792 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:36.695754 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4n6vl" podStartSLOduration=6.926979815 podStartE2EDuration="21.695739724s" totalImagesPullingTime="14.768759909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:36.695260268 +0000 UTC m=+22.790727059" watchObservedRunningTime="2026-09-10 04:41:36.695739724 +0000 UTC m=+22.791206462" Sep 10 04:41:37.182833 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.182807 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 04:41:37.312070 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.311928 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T04:41:37.182823889Z","UUID":"b407c391-a6c0-4bd6-a877-986840f3363b","Handler":null,"Name":"","Endpoint":""} Sep 10 04:41:37.314007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.313982 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 04:41:37.314121 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.314025 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 04:41:37.379409 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.379377 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:37.379559 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:37.379501 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:37.643520 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.643436 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" event={"ID":"abbbc61a-acc8-43d5-9e41-7054f79a9156","Type":"ContainerStarted","Data":"c05f923eabb364f7537a24bd87a80d88871ab47743dd9825bb6dcbc08029da6e"} Sep 10 04:41:37.645340 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.645308 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jtx8l" event={"ID":"4e431b1b-f695-46dc-862d-ed9f9590a5b7","Type":"ContainerStarted","Data":"bf2ebc69e1297b2f7b43ddda59fc97681b74c4db4d54a155dc675c5075d64cc9"} Sep 10 04:41:37.660284 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:37.660232 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jtx8l" podStartSLOduration=4.343592551 podStartE2EDuration="22.660218894s" totalImagesPullingTime="18.316626343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:37.659661333 +0000 UTC m=+23.755128092" watchObservedRunningTime="2026-09-10 04:41:37.660218894 +0000 UTC m=+23.755685631" Sep 10 04:41:38.379679 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:38.379574 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:38.379679 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:38.379597 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:38.379871 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:38.379721 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:38.379871 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:38.379822 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:38.650440 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:38.650358 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" event={"ID":"abbbc61a-acc8-43d5-9e41-7054f79a9156","Type":"ContainerStarted","Data":"7808e7de5253314e99126b3bcf5fa001d82324cdc62e809885fd1301d90e0d16"} Sep 10 04:41:38.654106 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:38.654079 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"05c0d80faaccd538f7a6118ff83ea0bde8704514534bbd640d6c11496f86e992"} Sep 10 04:41:38.667218 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:38.667177 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r88pn" podStartSLOduration=3.1850263659999998 podStartE2EDuration="23.667162703s" totalImagesPullingTime="20.482136337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:38.66705509 +0000 UTC m=+24.762521828" watchObservedRunningTime="2026-09-10 04:41:38.667162703 +0000 UTC m=+24.762629441" Sep 10 04:41:39.379553 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:39.379528 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:39.379739 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:39.379674 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:40.379084 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:40.379051 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:40.379567 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:40.379051 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:40.379567 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:40.379177 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:40.379567 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:40.379274 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:40.605024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:40.604994 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:40.605641 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:40.605616 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:40.659033 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:40.658967 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:40.659463 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:40.659444 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vhws4" Sep 10 04:41:41.379860 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:41.379629 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:41.380367 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:41.379985 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:41.664414 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:41.664378 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" event={"ID":"ac0af446-e525-4216-adfc-6188ecc148a5","Type":"ContainerStarted","Data":"8edae3822ece2d274d0c44337024bcf5d288c72880e06c8c8efa317d69607ff8"} Sep 10 04:41:41.664657 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:41.664630 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:41.666295 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:41.666269 2565 generic.go:356] "Generic (PLEG): container finished" podID="c417b44d-13b8-4555-83df-7e5229f14823" containerID="251f087d3a77e972f3821a413e1037bd897d6ee7c76e390431b32b8c3adab822" exitCode=0 Sep 10 04:41:41.666423 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:41.666342 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerDied","Data":"251f087d3a77e972f3821a413e1037bd897d6ee7c76e390431b32b8c3adab822"} Sep 10 04:41:41.679978 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:41.679959 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:41.692585 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:41.692548 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" podStartSLOduration=8.310684095 podStartE2EDuration="26.692535459s" totalImagesPullingTime="18.381851364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:41.692149349 +0000 UTC m=+27.787616095" watchObservedRunningTime="2026-09-10 04:41:41.692535459 +0000 UTC m=+27.788002196" Sep 10 04:41:42.383789 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:42.383707 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:42.384435 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:42.383822 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:42.384435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:42.383858 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:42.384435 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:42.383943 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:42.671446 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:42.671415 2565 generic.go:356] "Generic (PLEG): container finished" podID="c417b44d-13b8-4555-83df-7e5229f14823" containerID="a4ea0e6fa9a4ab1a783b193f390969da606cb19d0a79b86cdcc2003dbda0479c" exitCode=0 Sep 10 04:41:42.671586 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:42.671503 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerDied","Data":"a4ea0e6fa9a4ab1a783b193f390969da606cb19d0a79b86cdcc2003dbda0479c"} Sep 10 04:41:42.671675 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:42.671659 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:42.671909 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:42.671892 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:42.687693 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:42.687672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:41:43.151352 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.151170 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f2xkr"] Sep 10 04:41:43.151504 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.151414 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:43.151504 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:43.151492 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:43.154174 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.154148 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8pzhh"] Sep 10 04:41:43.154298 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.154231 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:43.154378 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:43.154302 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:43.154698 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.154677 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44mnf"] Sep 10 04:41:43.154785 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.154770 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:43.154898 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:43.154870 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:43.675872 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.675847 2565 generic.go:356] "Generic (PLEG): container finished" podID="c417b44d-13b8-4555-83df-7e5229f14823" containerID="1e7b5e9290d26a4fe1fc75e3e59bcd3bac556cc2a8c5e03642d3a052c36617a0" exitCode=0 Sep 10 04:41:43.676346 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:43.675926 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerDied","Data":"1e7b5e9290d26a4fe1fc75e3e59bcd3bac556cc2a8c5e03642d3a052c36617a0"} Sep 10 04:41:44.380984 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:44.380947 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:44.381149 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:44.381051 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:45.380039 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:45.380008 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:45.380503 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:45.380118 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:45.380503 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:45.380174 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:45.380503 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:45.380303 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:46.379947 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:46.379916 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:46.380103 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:46.380048 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:47.379223 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:47.379151 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:47.379376 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:47.379253 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8pzhh" podUID="94cbf17b-159d-43da-af57-639c03a0cabd" Sep 10 04:41:47.379376 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:47.379324 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:47.379502 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:47.379449 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44mnf" podUID="54664d95-d774-4a37-a7f1-eb37c2efb5ea" Sep 10 04:41:48.019227 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.019197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:48.019473 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.019234 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:48.019473 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.019337 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:48.019473 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.019349 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 04:41:48.019473 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.019363 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 04:41:48.019473 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.019375 2565 projected.go:196] Error preparing data for projected volume kube-api-access-6tgsn for pod openshift-network-diagnostics/network-check-target-8pzhh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:48.019473 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.019398 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs podName:54664d95-d774-4a37-a7f1-eb37c2efb5ea nodeName:}" failed. No retries permitted until 2026-09-10 04:42:20.019380686 +0000 UTC m=+66.114847423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs") pod "network-metrics-daemon-44mnf" (UID: "54664d95-d774-4a37-a7f1-eb37c2efb5ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 04:41:48.019473 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.019416 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn podName:94cbf17b-159d-43da-af57-639c03a0cabd nodeName:}" failed. No retries permitted until 2026-09-10 04:42:20.019407333 +0000 UTC m=+66.114874054 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tgsn" (UniqueName: "kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn") pod "network-check-target-8pzhh" (UID: "94cbf17b-159d-43da-af57-639c03a0cabd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 04:41:48.121246 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.121218 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:48.121384 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.121356 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:48.121488 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.121423 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret podName:f065d09c-3151-4335-bf0d-05452ac3aa70 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:20.121409092 +0000 UTC m=+66.216875808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret") pod "global-pull-secret-syncer-f2xkr" (UID: "f065d09c-3151-4335-bf0d-05452ac3aa70") : object "kube-system"/"original-pull-secret" not registered Sep 10 04:41:48.380034 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.380009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:48.380195 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:48.380109 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2xkr" podUID="f065d09c-3151-4335-bf0d-05452ac3aa70" Sep 10 04:41:48.690523 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.690487 2565 generic.go:356] "Generic (PLEG): container finished" podID="c417b44d-13b8-4555-83df-7e5229f14823" containerID="ed0dc118b19124a3e3a8f502aaf7ec3a4b5debb40d06157ae69117d6f794ec4a" exitCode=0 Sep 10 04:41:48.690725 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.690532 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerDied","Data":"ed0dc118b19124a3e3a8f502aaf7ec3a4b5debb40d06157ae69117d6f794ec4a"} Sep 10 04:41:48.694220 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.694199 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-230.ec2.internal" event="NodeReady" Sep 10 04:41:48.694327 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.694318 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 04:41:48.735974 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.735939 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-579784f65b-ncxmw"] Sep 10 04:41:48.757415 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.757390 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w82t5"] Sep 10 04:41:48.758694 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.758283 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.761119 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.760959 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 04:41:48.761119 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.760972 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 04:41:48.761119 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.761000 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z8h7d\"" Sep 10 04:41:48.761307 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.761277 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 04:41:48.775760 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.775728 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-29hct"] Sep 10 04:41:48.775857 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.775814 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:48.778086 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.778064 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 04:41:48.778288 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.778269 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cccd6\"" Sep 10 04:41:48.778825 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.778807 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 04:41:48.778911 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.778815 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 04:41:48.791863 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.791844 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-579784f65b-ncxmw"] Sep 10 04:41:48.791962 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.791870 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w82t5"] Sep 10 04:41:48.791962 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.791887 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-29hct"] Sep 10 04:41:48.791962 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.791952 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:48.794317 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.794298 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 04:41:48.794787 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.794766 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 04:41:48.794787 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.794767 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kvrb6\"" Sep 10 04:41:48.794918 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.794870 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 04:41:48.928195 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928168 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-bound-sa-token\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.928310 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:48.928310 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928249 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwj5m\" (UniqueName: \"kubernetes.io/projected/f1f53af7-3313-491c-829d-3658b362fae2-kube-api-access-vwj5m\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:48.928394 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928336 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh58l\" (UniqueName: \"kubernetes.io/projected/4bdee842-eefe-4d76-901f-15838d2e279b-kube-api-access-lh58l\") pod \"ingress-canary-29hct\" (UID: \"4bdee842-eefe-4d76-901f-15838d2e279b\") " pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:48.928394 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-trusted-ca\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.928491 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff5bcfc-8223-407f-a425-18854d184ffb-ca-trust-extracted\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.928491 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928455 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f53af7-3313-491c-829d-3658b362fae2-config-volume\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:48.928594 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928490 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4bdee842-eefe-4d76-901f-15838d2e279b-cert\") pod \"ingress-canary-29hct\" (UID: \"4bdee842-eefe-4d76-901f-15838d2e279b\") " pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:48.928594 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928521 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.928594 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928543 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f53af7-3313-491c-829d-3658b362fae2-tmp-dir\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:48.928594 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-certificates\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.928781 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928618 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-installation-pull-secrets\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.928781 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928649 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-image-registry-private-configuration\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:48.928781 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:48.928678 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfd6v\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-kube-api-access-wfd6v\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.029530 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029505 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-bound-sa-token\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029548 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwj5m\" (UniqueName: \"kubernetes.io/projected/f1f53af7-3313-491c-829d-3658b362fae2-kube-api-access-vwj5m\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029663 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh58l\" (UniqueName: \"kubernetes.io/projected/4bdee842-eefe-4d76-901f-15838d2e279b-kube-api-access-lh58l\") pod \"ingress-canary-29hct\" (UID: \"4bdee842-eefe-4d76-901f-15838d2e279b\") " pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029703 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-trusted-ca\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029723 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff5bcfc-8223-407f-a425-18854d184ffb-ca-trust-extracted\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.029671 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029751 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f53af7-3313-491c-829d-3658b362fae2-config-volume\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029781 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4bdee842-eefe-4d76-901f-15838d2e279b-cert\") pod \"ingress-canary-29hct\" (UID: \"4bdee842-eefe-4d76-901f-15838d2e279b\") " pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.029804 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls podName:f1f53af7-3313-491c-829d-3658b362fae2 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:49.529783904 +0000 UTC m=+35.625250631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls") pod "dns-default-w82t5" (UID: "f1f53af7-3313-491c-829d-3658b362fae2") : secret "dns-default-metrics-tls" not found Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029838 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.029858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029863 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f53af7-3313-491c-829d-3658b362fae2-tmp-dir\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-certificates\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-installation-pull-secrets\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.029926 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.029941 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579784f65b-ncxmw: secret "image-registry-tls" not found Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029947 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-image-registry-private-configuration\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.029973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfd6v\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-kube-api-access-wfd6v\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.029998 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls podName:9ff5bcfc-8223-407f-a425-18854d184ffb nodeName:}" failed. No retries permitted until 2026-09-10 04:41:49.52998103 +0000 UTC m=+35.625447745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls") pod "image-registry-579784f65b-ncxmw" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb") : secret "image-registry-tls" not found Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.030181 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff5bcfc-8223-407f-a425-18854d184ffb-ca-trust-extracted\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.030185 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f53af7-3313-491c-829d-3658b362fae2-tmp-dir\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.030399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.030373 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f1f53af7-3313-491c-829d-3658b362fae2-config-volume\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.030911 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.030582 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-certificates\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.031003 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.030983 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-trusted-ca\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.033931 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.033908 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-installation-pull-secrets\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.034014 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.033969 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-image-registry-private-configuration\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.034084 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.034018 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4bdee842-eefe-4d76-901f-15838d2e279b-cert\") pod \"ingress-canary-29hct\" (UID: \"4bdee842-eefe-4d76-901f-15838d2e279b\") " pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:49.038455 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.038142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfd6v\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-kube-api-access-wfd6v\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.038558 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.038448 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwj5m\" (UniqueName: \"kubernetes.io/projected/f1f53af7-3313-491c-829d-3658b362fae2-kube-api-access-vwj5m\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.039103 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.039080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh58l\" (UniqueName: \"kubernetes.io/projected/4bdee842-eefe-4d76-901f-15838d2e279b-kube-api-access-lh58l\") pod \"ingress-canary-29hct\" (UID: \"4bdee842-eefe-4d76-901f-15838d2e279b\") " pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:49.039194 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.039099 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-bound-sa-token\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.109729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.109709 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-29hct" Sep 10 04:41:49.379543 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.379512 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:41:49.379695 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.379629 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:41:49.383015 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.382990 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 04:41:49.383133 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.383024 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 04:41:49.383200 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.383158 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mwmz7\"" Sep 10 04:41:49.383275 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.383259 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l2xvl\"" Sep 10 04:41:49.383335 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.383327 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 04:41:49.431673 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.431652 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-29hct"] Sep 10 04:41:49.436317 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:41:49.436296 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdee842_eefe_4d76_901f_15838d2e279b.slice/crio-e85449d8d44770c8edc4e06aa45f6a4d28517bc2f7bf56c718409f13f254fa89 WatchSource:0}: Error finding container e85449d8d44770c8edc4e06aa45f6a4d28517bc2f7bf56c718409f13f254fa89: Status 404 returned error can't find the container with id e85449d8d44770c8edc4e06aa45f6a4d28517bc2f7bf56c718409f13f254fa89 Sep 10 04:41:49.534173 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.534114 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:49.534342 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.534174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:49.534342 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.534265 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 04:41:49.534342 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.534292 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 04:41:49.534342 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.534310 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579784f65b-ncxmw: secret "image-registry-tls" not found Sep 10 04:41:49.534342 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.534325 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls podName:f1f53af7-3313-491c-829d-3658b362fae2 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:50.534311884 +0000 UTC m=+36.629778599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls") pod "dns-default-w82t5" (UID: "f1f53af7-3313-491c-829d-3658b362fae2") : secret "dns-default-metrics-tls" not found Sep 10 04:41:49.534528 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:49.534369 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls podName:9ff5bcfc-8223-407f-a425-18854d184ffb nodeName:}" failed. No retries permitted until 2026-09-10 04:41:50.534349545 +0000 UTC m=+36.629816278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls") pod "image-registry-579784f65b-ncxmw" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb") : secret "image-registry-tls" not found Sep 10 04:41:49.696103 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.696072 2565 generic.go:356] "Generic (PLEG): container finished" podID="c417b44d-13b8-4555-83df-7e5229f14823" containerID="a5d4e4d7b9639948da759626a209f9de2e27df0f1c52d6935ba9b5fc4e4975fd" exitCode=0 Sep 10 04:41:49.696251 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.696153 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerDied","Data":"a5d4e4d7b9639948da759626a209f9de2e27df0f1c52d6935ba9b5fc4e4975fd"} Sep 10 04:41:49.697384 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:49.697363 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-29hct" event={"ID":"4bdee842-eefe-4d76-901f-15838d2e279b","Type":"ContainerStarted","Data":"e85449d8d44770c8edc4e06aa45f6a4d28517bc2f7bf56c718409f13f254fa89"} Sep 10 04:41:50.379356 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:50.379096 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:41:50.382362 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:50.382339 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 04:41:50.542097 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:50.542062 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:50.542249 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:50.542148 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:50.542249 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:50.542227 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 04:41:50.542249 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:50.542246 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 04:41:50.542415 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:50.542264 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579784f65b-ncxmw: secret "image-registry-tls" not found Sep 10 04:41:50.542415 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:50.542312 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls podName:f1f53af7-3313-491c-829d-3658b362fae2 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:52.542288447 +0000 UTC m=+38.637755173 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls") pod "dns-default-w82t5" (UID: "f1f53af7-3313-491c-829d-3658b362fae2") : secret "dns-default-metrics-tls" not found Sep 10 04:41:50.542415 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:50.542332 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls podName:9ff5bcfc-8223-407f-a425-18854d184ffb nodeName:}" failed. No retries permitted until 2026-09-10 04:41:52.542323246 +0000 UTC m=+38.637789966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls") pod "image-registry-579784f65b-ncxmw" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb") : secret "image-registry-tls" not found Sep 10 04:41:50.703892 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:50.703847 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" event={"ID":"c417b44d-13b8-4555-83df-7e5229f14823","Type":"ContainerStarted","Data":"f56d695147241e5c72713874a8e194683267bf5026d8321f0bfa93a3cf5806b9"} Sep 10 04:41:50.726769 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:50.726709 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mrd8j" podStartSLOduration=6.933220212 podStartE2EDuration="35.72669005s" totalImagesPullingTime="28.793469838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:50.725077086 +0000 UTC m=+36.820543866" watchObservedRunningTime="2026-09-10 04:41:50.72669005 +0000 UTC m=+36.822156791" Sep 10 04:41:52.554693 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:52.554653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:52.555112 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:52.554725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:52.555112 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:52.554812 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 04:41:52.555112 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:52.554878 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls podName:f1f53af7-3313-491c-829d-3658b362fae2 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:56.554862499 +0000 UTC m=+42.650329213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls") pod "dns-default-w82t5" (UID: "f1f53af7-3313-491c-829d-3658b362fae2") : secret "dns-default-metrics-tls" not found Sep 10 04:41:52.555112 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:52.554817 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 04:41:52.555112 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:52.554917 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579784f65b-ncxmw: secret "image-registry-tls" not found Sep 10 04:41:52.555112 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:52.554958 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls podName:9ff5bcfc-8223-407f-a425-18854d184ffb nodeName:}" failed. No retries permitted until 2026-09-10 04:41:56.554946716 +0000 UTC m=+42.650413431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls") pod "image-registry-579784f65b-ncxmw" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb") : secret "image-registry-tls" not found Sep 10 04:41:52.710468 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:52.710439 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-29hct" event={"ID":"4bdee842-eefe-4d76-901f-15838d2e279b","Type":"ContainerStarted","Data":"bc6ca87fa372769c54d4345571eca549918733aae54cd622c0ad4a814de8d513"} Sep 10 04:41:52.727717 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:52.727673 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-29hct" podStartSLOduration=2.177161675 podStartE2EDuration="4.727659082s" totalImagesPullingTime="2.550497407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:41:52.726461127 +0000 UTC m=+38.821927864" watchObservedRunningTime="2026-09-10 04:41:52.727659082 +0000 UTC m=+38.823125818" Sep 10 04:41:56.004793 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.004759 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mgxh2"] Sep 10 04:41:56.032688 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.032650 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mgxh2"] Sep 10 04:41:56.032823 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.032784 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.035672 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.035649 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 04:41:56.035986 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.035966 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 04:41:56.036094 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.036056 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sm5zg\"" Sep 10 04:41:56.036157 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.036058 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 04:41:56.036211 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.036134 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 04:41:56.181651 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.181588 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.181829 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.181672 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmwdw\" (UniqueName: \"kubernetes.io/projected/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-api-access-gmwdw\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.181829 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.181720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4896bd2-5b68-4f2e-be91-88c0c7727440-data-volume\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.181829 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.181751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.181829 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.181773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4896bd2-5b68-4f2e-be91-88c0c7727440-crio-socket\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282134 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.282040 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4896bd2-5b68-4f2e-be91-88c0c7727440-data-volume\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282134 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.282104 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282134 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.282133 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4896bd2-5b68-4f2e-be91-88c0c7727440-crio-socket\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282343 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.282253 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 04:41:56.282343 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.282308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282343 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.282324 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:56.782307567 +0000 UTC m=+42.877774283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : secret "insights-runtime-extractor-tls" not found Sep 10 04:41:56.282451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.282349 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4896bd2-5b68-4f2e-be91-88c0c7727440-crio-socket\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282451 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.282359 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 04:41:56.282451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.282362 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmwdw\" (UniqueName: \"kubernetes.io/projected/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-api-access-gmwdw\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.282349 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4896bd2-5b68-4f2e-be91-88c0c7727440-data-volume\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.282451 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.282412 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:56.782398598 +0000 UTC m=+42.877865314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : configmap "kube-rbac-proxy" not found Sep 10 04:41:56.293513 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.293483 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmwdw\" (UniqueName: \"kubernetes.io/projected/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-api-access-gmwdw\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.584313 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.584226 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:41:56.584313 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.584295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:41:56.584496 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.584394 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 04:41:56.584496 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.584421 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579784f65b-ncxmw: secret "image-registry-tls" not found Sep 10 04:41:56.584496 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.584431 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 04:41:56.584496 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.584497 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls podName:f1f53af7-3313-491c-829d-3658b362fae2 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:04.584475061 +0000 UTC m=+50.679941790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls") pod "dns-default-w82t5" (UID: "f1f53af7-3313-491c-829d-3658b362fae2") : secret "dns-default-metrics-tls" not found Sep 10 04:41:56.584657 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.584514 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls podName:9ff5bcfc-8223-407f-a425-18854d184ffb nodeName:}" failed. No retries permitted until 2026-09-10 04:42:04.584505517 +0000 UTC m=+50.679972233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls") pod "image-registry-579784f65b-ncxmw" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb") : secret "image-registry-tls" not found Sep 10 04:41:56.785738 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.785700 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.785907 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:56.785775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:56.785907 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.785819 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 04:41:56.785907 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.785869 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 04:41:56.785907 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.785886 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:57.785871159 +0000 UTC m=+43.881337873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : configmap "kube-rbac-proxy" not found Sep 10 04:41:56.786041 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:56.785922 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:57.785911832 +0000 UTC m=+43.881378547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : secret "insights-runtime-extractor-tls" not found Sep 10 04:41:57.791909 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:57.791872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:57.792282 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:57.791939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:57.792282 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:57.791994 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 04:41:57.792282 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:57.792030 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 04:41:57.792282 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:57.792083 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:59.792062523 +0000 UTC m=+45.887529253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : configmap "kube-rbac-proxy" not found Sep 10 04:41:57.792282 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:57.792099 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:41:59.792092759 +0000 UTC m=+45.887559473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : secret "insights-runtime-extractor-tls" not found Sep 10 04:41:57.826513 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:57.826490 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4n6vl_7d2881f6-4630-4769-90e6-ea5c9af3f4d8/dns-node-resolver/0.log" Sep 10 04:41:59.026411 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:59.026382 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bllrf_a308358a-d20b-40d7-b9bc-a95c79d0c32a/node-ca/0.log" Sep 10 04:41:59.626680 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:59.626650 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-29hct_4bdee842-eefe-4d76-901f-15838d2e279b/serve-healthcheck-canary/0.log" Sep 10 04:41:59.807448 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:59.807412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:59.807661 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:41:59.807478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:41:59.807661 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:59.807537 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 04:41:59.807661 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:59.807563 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 04:41:59.807661 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:59.807625 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:03.807592315 +0000 UTC m=+49.903059034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : configmap "kube-rbac-proxy" not found Sep 10 04:41:59.807661 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:41:59.807644 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:03.807635126 +0000 UTC m=+49.903101844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : secret "insights-runtime-extractor-tls" not found Sep 10 04:42:03.836818 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:03.836769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:42:03.837265 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:03.836848 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:42:03.837265 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:03.836917 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 04:42:03.837265 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:03.836986 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls podName:e4896bd2-5b68-4f2e-be91-88c0c7727440 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:11.836970451 +0000 UTC m=+57.932437166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgxh2" (UID: "e4896bd2-5b68-4f2e-be91-88c0c7727440") : secret "insights-runtime-extractor-tls" not found Sep 10 04:42:03.837376 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:03.837323 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4896bd2-5b68-4f2e-be91-88c0c7727440-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:42:04.641041 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:04.641003 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:42:04.641247 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:04.641100 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:42:04.641247 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:04.641179 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 04:42:04.641247 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:04.641231 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 04:42:04.641247 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:04.641242 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls podName:f1f53af7-3313-491c-829d-3658b362fae2 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:20.64122543 +0000 UTC m=+66.736692151 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls") pod "dns-default-w82t5" (UID: "f1f53af7-3313-491c-829d-3658b362fae2") : secret "dns-default-metrics-tls" not found Sep 10 04:42:04.641247 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:04.641248 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579784f65b-ncxmw: secret "image-registry-tls" not found Sep 10 04:42:04.641458 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:04.641294 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls podName:9ff5bcfc-8223-407f-a425-18854d184ffb nodeName:}" failed. No retries permitted until 2026-09-10 04:42:20.641278974 +0000 UTC m=+66.736745711 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls") pod "image-registry-579784f65b-ncxmw" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb") : secret "image-registry-tls" not found Sep 10 04:42:11.897914 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:11.897876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:42:11.900085 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:11.900061 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4896bd2-5b68-4f2e-be91-88c0c7727440-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgxh2\" (UID: \"e4896bd2-5b68-4f2e-be91-88c0c7727440\") " pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:42:11.941137 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:11.941110 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mgxh2" Sep 10 04:42:12.233943 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:12.233914 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mgxh2"] Sep 10 04:42:12.237279 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:12.237254 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4896bd2_5b68_4f2e_be91_88c0c7727440.slice/crio-c13c59f5976ef5d60d4aa4a3970c0299aa1d98967afc455a9cf99803cf3096b9 WatchSource:0}: Error finding container c13c59f5976ef5d60d4aa4a3970c0299aa1d98967afc455a9cf99803cf3096b9: Status 404 returned error can't find the container with id c13c59f5976ef5d60d4aa4a3970c0299aa1d98967afc455a9cf99803cf3096b9 Sep 10 04:42:12.757907 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:12.757873 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgxh2" event={"ID":"e4896bd2-5b68-4f2e-be91-88c0c7727440","Type":"ContainerStarted","Data":"bd45edf2d47c026df1961b298ad7a9a452ea53ed47d5d84a45a69d53360ad724"} Sep 10 04:42:12.758063 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:12.757929 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgxh2" event={"ID":"e4896bd2-5b68-4f2e-be91-88c0c7727440","Type":"ContainerStarted","Data":"c13c59f5976ef5d60d4aa4a3970c0299aa1d98967afc455a9cf99803cf3096b9"} Sep 10 04:42:13.761822 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:13.761784 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgxh2" event={"ID":"e4896bd2-5b68-4f2e-be91-88c0c7727440","Type":"ContainerStarted","Data":"986080e2da990911fb8da1b52638825398269c0a91865dcf1937cc6b84db67bf"} Sep 10 04:42:14.689806 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:14.689776 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9cp4" Sep 10 04:42:14.766620 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:14.766568 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgxh2" event={"ID":"e4896bd2-5b68-4f2e-be91-88c0c7727440","Type":"ContainerStarted","Data":"174df96d398ffa7c5fc6aed99933bd80099cf232ff7d3ecf085f038d71568567"} Sep 10 04:42:14.786352 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:14.784829 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mgxh2" podStartSLOduration=18.081329782 podStartE2EDuration="19.784808515s" totalImagesPullingTime="1.703478733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:14.783561953 +0000 UTC m=+60.879028694" watchObservedRunningTime="2026-09-10 04:42:14.784808515 +0000 UTC m=+60.880275254" Sep 10 04:42:19.651868 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.651832 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r"] Sep 10 04:42:19.681247 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.681217 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-579784f65b-ncxmw"] Sep 10 04:42:19.681412 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.681258 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r"] Sep 10 04:42:19.681412 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.681351 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.681530 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:19.681411 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-579784f65b-ncxmw" podUID="9ff5bcfc-8223-407f-a425-18854d184ffb" Sep 10 04:42:19.684368 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.684349 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 04:42:19.684756 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.684732 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 10 04:42:19.684874 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.684758 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 04:42:19.684874 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.684776 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 04:42:19.760986 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.760953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd6e1e48-23c9-46cc-90c0-b176df41fc47-tmp\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.760986 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.760990 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bd6e1e48-23c9-46cc-90c0-b176df41fc47-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.761194 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.761025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cptwr\" (UniqueName: \"kubernetes.io/projected/bd6e1e48-23c9-46cc-90c0-b176df41fc47-kube-api-access-cptwr\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.769481 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.769451 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56775dcb96-hkm7j"] Sep 10 04:42:19.771242 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.771224 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.778173 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.778153 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:42:19.782421 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.782406 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:42:19.793336 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.793314 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56775dcb96-hkm7j"] Sep 10 04:42:19.861376 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861344 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff5bcfc-8223-407f-a425-18854d184ffb-ca-trust-extracted\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:19.861376 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861383 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-bound-sa-token\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:19.861584 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861408 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-installation-pull-secrets\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:19.861584 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861431 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-image-registry-private-configuration\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:19.861584 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861456 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-certificates\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:19.861584 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861481 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfd6v\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-kube-api-access-wfd6v\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:19.861584 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861534 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-trusted-ca\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:19.861850 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861627 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff5bcfc-8223-407f-a425-18854d184ffb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 04:42:19.861850 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861640 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4qcw\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-kube-api-access-z4qcw\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.861850 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-bound-sa-token\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.861850 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5295dded-ad87-480c-9126-7ca5867708b0-ca-trust-extracted\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.861850 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5295dded-ad87-480c-9126-7ca5867708b0-trusted-ca\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.862095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5295dded-ad87-480c-9126-7ca5867708b0-registry-certificates\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.862095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861894 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:42:19.862095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861909 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-registry-tls\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.862095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861948 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5295dded-ad87-480c-9126-7ca5867708b0-image-registry-private-configuration\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.862095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.861993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd6e1e48-23c9-46cc-90c0-b176df41fc47-tmp\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.862095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.862028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bd6e1e48-23c9-46cc-90c0-b176df41fc47-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.862370 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.862117 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cptwr\" (UniqueName: \"kubernetes.io/projected/bd6e1e48-23c9-46cc-90c0-b176df41fc47-kube-api-access-cptwr\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.862370 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.862144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5295dded-ad87-480c-9126-7ca5867708b0-installation-pull-secrets\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.862370 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.862186 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-certificates\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:19.862370 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.862203 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff5bcfc-8223-407f-a425-18854d184ffb-ca-trust-extracted\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:19.862370 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.862267 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:42:19.862566 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.862374 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd6e1e48-23c9-46cc-90c0-b176df41fc47-tmp\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.863905 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.863873 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:42:19.864021 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.863967 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:42:19.864086 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.864064 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-kube-api-access-wfd6v" (OuterVolumeSpecName: "kube-api-access-wfd6v") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "kube-api-access-wfd6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:42:19.864130 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.864083 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:42:19.864644 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.864627 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bd6e1e48-23c9-46cc-90c0-b176df41fc47-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.876387 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.876362 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cptwr\" (UniqueName: \"kubernetes.io/projected/bd6e1e48-23c9-46cc-90c0-b176df41fc47-kube-api-access-cptwr\") pod \"klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r\" (UID: \"bd6e1e48-23c9-46cc-90c0-b176df41fc47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:19.963180 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963145 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-registry-tls\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963342 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963191 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5295dded-ad87-480c-9126-7ca5867708b0-image-registry-private-configuration\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963342 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963236 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5295dded-ad87-480c-9126-7ca5867708b0-installation-pull-secrets\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963342 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963252 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4qcw\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-kube-api-access-z4qcw\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963342 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-bound-sa-token\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963555 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963435 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5295dded-ad87-480c-9126-7ca5867708b0-ca-trust-extracted\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963555 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5295dded-ad87-480c-9126-7ca5867708b0-trusted-ca\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963555 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5295dded-ad87-480c-9126-7ca5867708b0-registry-certificates\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.963746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963621 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-image-registry-private-configuration\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:19.963746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963639 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfd6v\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-kube-api-access-wfd6v\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:19.963746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963654 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff5bcfc-8223-407f-a425-18854d184ffb-installation-pull-secrets\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:19.963746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963675 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff5bcfc-8223-407f-a425-18854d184ffb-trusted-ca\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:19.963746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963693 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-bound-sa-token\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:19.963977 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.963879 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5295dded-ad87-480c-9126-7ca5867708b0-ca-trust-extracted\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.964293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.964269 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5295dded-ad87-480c-9126-7ca5867708b0-registry-certificates\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.964361 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.964344 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5295dded-ad87-480c-9126-7ca5867708b0-trusted-ca\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.965667 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.965648 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5295dded-ad87-480c-9126-7ca5867708b0-image-registry-private-configuration\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.966056 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.966033 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-registry-tls\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.966172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.966142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5295dded-ad87-480c-9126-7ca5867708b0-installation-pull-secrets\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.972077 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.972053 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-bound-sa-token\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.972591 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.972576 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4qcw\" (UniqueName: \"kubernetes.io/projected/5295dded-ad87-480c-9126-7ca5867708b0-kube-api-access-z4qcw\") pod \"image-registry-56775dcb96-hkm7j\" (UID: \"5295dded-ad87-480c-9126-7ca5867708b0\") " pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:19.990596 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:19.990575 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:20.064193 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.064163 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:42:20.064327 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.064200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:42:20.066656 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.066637 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 04:42:20.066727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.066712 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 04:42:20.076848 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.076825 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54664d95-d774-4a37-a7f1-eb37c2efb5ea-metrics-certs\") pod \"network-metrics-daemon-44mnf\" (UID: \"54664d95-d774-4a37-a7f1-eb37c2efb5ea\") " pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:42:20.077080 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.077066 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 04:42:20.081366 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.081352 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z8h7d\"" Sep 10 04:42:20.087180 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.087164 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tgsn\" (UniqueName: \"kubernetes.io/projected/94cbf17b-159d-43da-af57-639c03a0cabd-kube-api-access-6tgsn\") pod \"network-check-target-8pzhh\" (UID: \"94cbf17b-159d-43da-af57-639c03a0cabd\") " pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:42:20.089968 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.089953 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:20.164842 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.164812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:42:20.167207 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.167186 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 04:42:20.177097 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.177075 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f065d09c-3151-4335-bf0d-05452ac3aa70-original-pull-secret\") pod \"global-pull-secret-syncer-f2xkr\" (UID: \"f065d09c-3151-4335-bf0d-05452ac3aa70\") " pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:42:20.259193 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.259027 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r"] Sep 10 04:42:20.261748 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:20.261727 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6e1e48_23c9_46cc_90c0_b176df41fc47.slice/crio-f27773d80f9785e518a282c3647db3d722a2b9d96dad02483d9e9c7126633cd6 WatchSource:0}: Error finding container f27773d80f9785e518a282c3647db3d722a2b9d96dad02483d9e9c7126633cd6: Status 404 returned error can't find the container with id f27773d80f9785e518a282c3647db3d722a2b9d96dad02483d9e9c7126633cd6 Sep 10 04:42:20.291680 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.291656 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l2xvl\"" Sep 10 04:42:20.295771 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.295754 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mwmz7\"" Sep 10 04:42:20.299561 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.299547 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:42:20.304665 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.304649 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44mnf" Sep 10 04:42:20.359702 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.359673 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56775dcb96-hkm7j"] Sep 10 04:42:20.369382 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:20.368950 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5295dded_ad87_480c_9126_7ca5867708b0.slice/crio-d05da10a2d6a5442341696ea94cc33acaecc2b7328a2f7ffdb4ffc779ccea1d9 WatchSource:0}: Error finding container d05da10a2d6a5442341696ea94cc33acaecc2b7328a2f7ffdb4ffc779ccea1d9: Status 404 returned error can't find the container with id d05da10a2d6a5442341696ea94cc33acaecc2b7328a2f7ffdb4ffc779ccea1d9 Sep 10 04:42:20.390023 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.390000 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2xkr" Sep 10 04:42:20.568359 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.568267 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8pzhh"] Sep 10 04:42:20.571585 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:20.571557 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cbf17b_159d_43da_af57_639c03a0cabd.slice/crio-535ccf02be296a54a24eec1f4d472b948cc1a63e5b10facd3d92e94063f851e6 WatchSource:0}: Error finding container 535ccf02be296a54a24eec1f4d472b948cc1a63e5b10facd3d92e94063f851e6: Status 404 returned error can't find the container with id 535ccf02be296a54a24eec1f4d472b948cc1a63e5b10facd3d92e94063f851e6 Sep 10 04:42:20.588034 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.587885 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44mnf"] Sep 10 04:42:20.590428 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:20.590401 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54664d95_d774_4a37_a7f1_eb37c2efb5ea.slice/crio-8ce0ba390f018786d6f565969445bdd7ef5224a2930ef1dfd3963180eb2e9c4a WatchSource:0}: Error finding container 8ce0ba390f018786d6f565969445bdd7ef5224a2930ef1dfd3963180eb2e9c4a: Status 404 returned error can't find the container with id 8ce0ba390f018786d6f565969445bdd7ef5224a2930ef1dfd3963180eb2e9c4a Sep 10 04:42:20.660142 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.660103 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f2xkr"] Sep 10 04:42:20.664614 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:20.664578 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf065d09c_3151_4335_bf0d_05452ac3aa70.slice/crio-79be219f743ecf0f867d99f69cd7ff76f6ffdc5e8947722b44f57486b1012761 WatchSource:0}: Error finding container 79be219f743ecf0f867d99f69cd7ff76f6ffdc5e8947722b44f57486b1012761: Status 404 returned error can't find the container with id 79be219f743ecf0f867d99f69cd7ff76f6ffdc5e8947722b44f57486b1012761 Sep 10 04:42:20.670060 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.670039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:42:20.670135 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.670077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:42:20.672303 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.672277 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1f53af7-3313-491c-829d-3658b362fae2-metrics-tls\") pod \"dns-default-w82t5\" (UID: \"f1f53af7-3313-491c-829d-3658b362fae2\") " pod="openshift-dns/dns-default-w82t5" Sep 10 04:42:20.672567 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.672545 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"image-registry-579784f65b-ncxmw\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:42:20.770854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.770811 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") pod \"9ff5bcfc-8223-407f-a425-18854d184ffb\" (UID: \"9ff5bcfc-8223-407f-a425-18854d184ffb\") " Sep 10 04:42:20.772756 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.772729 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9ff5bcfc-8223-407f-a425-18854d184ffb" (UID: "9ff5bcfc-8223-407f-a425-18854d184ffb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:42:20.782295 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.782263 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8pzhh" event={"ID":"94cbf17b-159d-43da-af57-639c03a0cabd","Type":"ContainerStarted","Data":"535ccf02be296a54a24eec1f4d472b948cc1a63e5b10facd3d92e94063f851e6"} Sep 10 04:42:20.783492 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.783463 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44mnf" event={"ID":"54664d95-d774-4a37-a7f1-eb37c2efb5ea","Type":"ContainerStarted","Data":"8ce0ba390f018786d6f565969445bdd7ef5224a2930ef1dfd3963180eb2e9c4a"} Sep 10 04:42:20.784656 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.784627 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" event={"ID":"bd6e1e48-23c9-46cc-90c0-b176df41fc47","Type":"ContainerStarted","Data":"f27773d80f9785e518a282c3647db3d722a2b9d96dad02483d9e9c7126633cd6"} Sep 10 04:42:20.785786 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.785762 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f2xkr" event={"ID":"f065d09c-3151-4335-bf0d-05452ac3aa70","Type":"ContainerStarted","Data":"79be219f743ecf0f867d99f69cd7ff76f6ffdc5e8947722b44f57486b1012761"} Sep 10 04:42:20.787239 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.787208 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" event={"ID":"5295dded-ad87-480c-9126-7ca5867708b0","Type":"ContainerStarted","Data":"5aa14110cd4c471816f7900af64cd36c0ea95adc0635304f64ba5d6332e2b9b9"} Sep 10 04:42:20.787300 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.787245 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579784f65b-ncxmw" Sep 10 04:42:20.787300 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.787257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" event={"ID":"5295dded-ad87-480c-9126-7ca5867708b0","Type":"ContainerStarted","Data":"d05da10a2d6a5442341696ea94cc33acaecc2b7328a2f7ffdb4ffc779ccea1d9"} Sep 10 04:42:20.787625 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.787583 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:20.819613 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.819523 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-579784f65b-ncxmw"] Sep 10 04:42:20.824454 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.824431 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-579784f65b-ncxmw"] Sep 10 04:42:20.847992 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.847939 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" podStartSLOduration=1.847918631 podStartE2EDuration="1.847918631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:20.846598004 +0000 UTC m=+66.942064762" watchObservedRunningTime="2026-09-10 04:42:20.847918631 +0000 UTC m=+66.943385371" Sep 10 04:42:20.873345 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.871789 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff5bcfc-8223-407f-a425-18854d184ffb-registry-tls\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:42:20.906957 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.906880 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cccd6\"" Sep 10 04:42:20.914965 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:20.914937 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w82t5" Sep 10 04:42:21.208005 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:21.207926 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w82t5"] Sep 10 04:42:21.221733 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:21.221711 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f53af7_3313_491c_829d_3658b362fae2.slice/crio-9a5fb93e748217c3d272df3f9103c8e04310b04efcece9ba42eb15f263411435 WatchSource:0}: Error finding container 9a5fb93e748217c3d272df3f9103c8e04310b04efcece9ba42eb15f263411435: Status 404 returned error can't find the container with id 9a5fb93e748217c3d272df3f9103c8e04310b04efcece9ba42eb15f263411435 Sep 10 04:42:21.792033 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:21.791953 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w82t5" event={"ID":"f1f53af7-3313-491c-829d-3658b362fae2","Type":"ContainerStarted","Data":"9a5fb93e748217c3d272df3f9103c8e04310b04efcece9ba42eb15f263411435"} Sep 10 04:42:22.384703 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:22.384470 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ff5bcfc-8223-407f-a425-18854d184ffb" path="/var/lib/kubelet/pods/9ff5bcfc-8223-407f-a425-18854d184ffb/volumes" Sep 10 04:42:23.566283 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:23.566247 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl"] Sep 10 04:42:23.568658 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:23.568630 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:23.571218 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:23.571198 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 04:42:23.571326 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:23.571204 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jtbnl\"" Sep 10 04:42:23.577012 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:23.576465 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl"] Sep 10 04:42:23.699446 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:23.699347 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ab97a86-da91-49d2-ad05-abf5b791e314-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2r7gl\" (UID: \"5ab97a86-da91-49d2-ad05-abf5b791e314\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:23.799877 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:23.799844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ab97a86-da91-49d2-ad05-abf5b791e314-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2r7gl\" (UID: \"5ab97a86-da91-49d2-ad05-abf5b791e314\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:23.800031 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:23.800020 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 10 04:42:23.800113 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:23.800100 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ab97a86-da91-49d2-ad05-abf5b791e314-tls-certificates podName:5ab97a86-da91-49d2-ad05-abf5b791e314 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:24.300074977 +0000 UTC m=+70.395541698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5ab97a86-da91-49d2-ad05-abf5b791e314-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-2r7gl" (UID: "5ab97a86-da91-49d2-ad05-abf5b791e314") : secret "prometheus-operator-admission-webhook-tls" not found Sep 10 04:42:24.304597 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:24.304560 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ab97a86-da91-49d2-ad05-abf5b791e314-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2r7gl\" (UID: \"5ab97a86-da91-49d2-ad05-abf5b791e314\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:24.307248 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:24.307219 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ab97a86-da91-49d2-ad05-abf5b791e314-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2r7gl\" (UID: \"5ab97a86-da91-49d2-ad05-abf5b791e314\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:24.480648 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:24.480590 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:27.706833 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.706797 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl"] Sep 10 04:42:27.712301 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:27.712271 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab97a86_da91_49d2_ad05_abf5b791e314.slice/crio-3fb52c3944951fdde4cfa735c8523f7c11341d43afdcd2513711870c68af80ff WatchSource:0}: Error finding container 3fb52c3944951fdde4cfa735c8523f7c11341d43afdcd2513711870c68af80ff: Status 404 returned error can't find the container with id 3fb52c3944951fdde4cfa735c8523f7c11341d43afdcd2513711870c68af80ff Sep 10 04:42:27.813951 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.813917 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8pzhh" event={"ID":"94cbf17b-159d-43da-af57-639c03a0cabd","Type":"ContainerStarted","Data":"f6c2eb365f064d686a97c21b9bc896f35362f7e0712dbb5ffd0449c6e482b5a8"} Sep 10 04:42:27.814132 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.813999 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:42:27.815651 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.815630 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w82t5" event={"ID":"f1f53af7-3313-491c-829d-3658b362fae2","Type":"ContainerStarted","Data":"8ed2eb6ba468716dc104e625c10d35a58d8d0291f4cfaee2fd7b7780159572a8"} Sep 10 04:42:27.815779 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.815671 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w82t5" event={"ID":"f1f53af7-3313-491c-829d-3658b362fae2","Type":"ContainerStarted","Data":"c339c203dcc1a70108a9428cd932b4d89c73c385681c8a37675a303735f3e34a"} Sep 10 04:42:27.815779 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.815708 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w82t5" Sep 10 04:42:27.816761 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.816716 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" event={"ID":"5ab97a86-da91-49d2-ad05-abf5b791e314","Type":"ContainerStarted","Data":"3fb52c3944951fdde4cfa735c8523f7c11341d43afdcd2513711870c68af80ff"} Sep 10 04:42:27.818445 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.818426 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44mnf" event={"ID":"54664d95-d774-4a37-a7f1-eb37c2efb5ea","Type":"ContainerStarted","Data":"83660efddf3df73ad44fb5e1fb195957e1d56705c8c86ec69ad210e9af66e134"} Sep 10 04:42:27.818554 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.818467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44mnf" event={"ID":"54664d95-d774-4a37-a7f1-eb37c2efb5ea","Type":"ContainerStarted","Data":"89352607f40ed35688af53014a3b75bf715a5eae2e800586fb285a7d076b9828"} Sep 10 04:42:27.819790 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.819769 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" event={"ID":"bd6e1e48-23c9-46cc-90c0-b176df41fc47","Type":"ContainerStarted","Data":"96a0120f2816edd9864929f3a34faac3b7301e0f6c438eb7fa8834c5126f830b"} Sep 10 04:42:27.819977 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.819929 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:27.821695 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.821680 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" Sep 10 04:42:27.821776 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.821711 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f2xkr" event={"ID":"f065d09c-3151-4335-bf0d-05452ac3aa70","Type":"ContainerStarted","Data":"61c130a399b17d95b959b9cf6be1d9c2748c783cd3c47c12e43d8a978cdcf478"} Sep 10 04:42:27.828842 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.828804 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8pzhh" podStartSLOduration=66.036456507 podStartE2EDuration="1m12.828792758s" totalImagesPullingTime="6.792336251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:27.828679797 +0000 UTC m=+73.924146559" watchObservedRunningTime="2026-09-10 04:42:27.828792758 +0000 UTC m=+73.924259486" Sep 10 04:42:27.846411 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.846336 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-44mnf" podStartSLOduration=66.143432861 podStartE2EDuration="1m12.846321091s" totalImagesPullingTime="6.70288823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:27.845978693 +0000 UTC m=+73.941445456" watchObservedRunningTime="2026-09-10 04:42:27.846321091 +0000 UTC m=+73.941787830" Sep 10 04:42:27.864683 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.864649 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5b4b8fdf-wbp8r" podStartSLOduration=1.743454004 podStartE2EDuration="8.864635657s" totalImagesPullingTime="7.121181653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:27.864164146 +0000 UTC m=+73.959630916" watchObservedRunningTime="2026-09-10 04:42:27.864635657 +0000 UTC m=+73.960102394" Sep 10 04:42:27.879141 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.879100 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f2xkr" podStartSLOduration=66.16140307 podStartE2EDuration="1m12.879087903s" totalImagesPullingTime="6.717684833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:27.878668841 +0000 UTC m=+73.974135597" watchObservedRunningTime="2026-09-10 04:42:27.879087903 +0000 UTC m=+73.974554633" Sep 10 04:42:27.895477 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:27.895441 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w82t5" podStartSLOduration=33.755726436 podStartE2EDuration="39.895429059s" totalImagesPullingTime="6.139702623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:27.894782165 +0000 UTC m=+73.990248939" watchObservedRunningTime="2026-09-10 04:42:27.895429059 +0000 UTC m=+73.990895796" Sep 10 04:42:28.832402 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:28.832362 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" event={"ID":"5ab97a86-da91-49d2-ad05-abf5b791e314","Type":"ContainerStarted","Data":"b40b87c9f2872dfb53a3bb3baa86d92a0d47ae7a9a110db295aa60c4127a0ccd"} Sep 10 04:42:28.832902 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:28.832557 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:28.837266 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:28.837243 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" Sep 10 04:42:28.848027 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:28.847988 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2r7gl" podStartSLOduration=4.8688154390000005 podStartE2EDuration="5.847972241s" totalImagesPullingTime="979.156802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:28.846557524 +0000 UTC m=+74.942024261" watchObservedRunningTime="2026-09-10 04:42:28.847972241 +0000 UTC m=+74.943438979" Sep 10 04:42:29.642963 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.642930 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pnvgw"] Sep 10 04:42:29.645955 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.645939 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.648471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.648445 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 10 04:42:29.648600 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.648567 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 04:42:29.649431 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.649407 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 10 04:42:29.649512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.649411 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wvk49\"" Sep 10 04:42:29.649512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.649503 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 04:42:29.649760 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.649742 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 04:42:29.657402 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.657382 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pnvgw"] Sep 10 04:42:29.760487 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.760453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05798918-4dae-42e7-8af3-b4cee9cfc2d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.760698 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.760495 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9945\" (UniqueName: \"kubernetes.io/projected/05798918-4dae-42e7-8af3-b4cee9cfc2d1-kube-api-access-z9945\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.760698 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.760521 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05798918-4dae-42e7-8af3-b4cee9cfc2d1-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.760698 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.760642 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05798918-4dae-42e7-8af3-b4cee9cfc2d1-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.861815 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.861785 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05798918-4dae-42e7-8af3-b4cee9cfc2d1-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.862138 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.861835 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05798918-4dae-42e7-8af3-b4cee9cfc2d1-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.862138 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.861871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05798918-4dae-42e7-8af3-b4cee9cfc2d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.862138 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.861892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9945\" (UniqueName: \"kubernetes.io/projected/05798918-4dae-42e7-8af3-b4cee9cfc2d1-kube-api-access-z9945\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.862521 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.862499 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05798918-4dae-42e7-8af3-b4cee9cfc2d1-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.864096 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.864076 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05798918-4dae-42e7-8af3-b4cee9cfc2d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.864188 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.864167 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05798918-4dae-42e7-8af3-b4cee9cfc2d1-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.873653 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.873632 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9945\" (UniqueName: \"kubernetes.io/projected/05798918-4dae-42e7-8af3-b4cee9cfc2d1-kube-api-access-z9945\") pod \"prometheus-operator-6f4946db56-pnvgw\" (UID: \"05798918-4dae-42e7-8af3-b4cee9cfc2d1\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:29.954704 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:29.954684 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" Sep 10 04:42:30.222065 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:30.221990 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pnvgw"] Sep 10 04:42:30.224996 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:30.224972 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05798918_4dae_42e7_8af3_b4cee9cfc2d1.slice/crio-3c1962c918a8bc99b16acc3ee6be426a0ba8ef573a887991d3eab40536d262f6 WatchSource:0}: Error finding container 3c1962c918a8bc99b16acc3ee6be426a0ba8ef573a887991d3eab40536d262f6: Status 404 returned error can't find the container with id 3c1962c918a8bc99b16acc3ee6be426a0ba8ef573a887991d3eab40536d262f6 Sep 10 04:42:30.840353 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:30.840317 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" event={"ID":"05798918-4dae-42e7-8af3-b4cee9cfc2d1","Type":"ContainerStarted","Data":"3c1962c918a8bc99b16acc3ee6be426a0ba8ef573a887991d3eab40536d262f6"} Sep 10 04:42:31.845475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:31.845435 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" event={"ID":"05798918-4dae-42e7-8af3-b4cee9cfc2d1","Type":"ContainerStarted","Data":"91b207e88a50d093d1e7db8c34957c7422ab6ecb6ffd6d1f315b543f616866cf"} Sep 10 04:42:31.845860 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:31.845505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" event={"ID":"05798918-4dae-42e7-8af3-b4cee9cfc2d1","Type":"ContainerStarted","Data":"cbf4240dbf044912fced0d84fcfbe0c280134c18039a5550edb67440df580798"} Sep 10 04:42:31.864746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:31.864690 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-pnvgw" podStartSLOduration=1.7386695890000001 podStartE2EDuration="2.864673468s" totalImagesPullingTime="1.126003879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:31.862989523 +0000 UTC m=+77.958456260" watchObservedRunningTime="2026-09-10 04:42:31.864673468 +0000 UTC m=+77.960140205" Sep 10 04:42:34.000310 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.000276 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2"] Sep 10 04:42:34.003706 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.003687 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.006095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.006076 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 04:42:34.006179 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.006102 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 04:42:34.006224 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.006208 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-h7tl2\"" Sep 10 04:42:34.015443 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.015421 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2"] Sep 10 04:42:34.031946 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.031916 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv"] Sep 10 04:42:34.034962 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.034945 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s9vs6"] Sep 10 04:42:34.035075 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.035059 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.037519 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.037501 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 04:42:34.037706 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.037693 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 04:42:34.037919 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.037900 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pbctw\"" Sep 10 04:42:34.038052 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.038030 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.038883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.038864 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 04:42:34.041338 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.041318 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 04:42:34.041472 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.041384 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 04:42:34.041472 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.041449 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 04:42:34.041717 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.041516 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mh452\"" Sep 10 04:42:34.051258 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.051238 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv"] Sep 10 04:42:34.094351 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.094316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.094528 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.094368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3363df8b-023e-4b28-a595-8dd9f056f9da-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.094528 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.094425 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkmh9\" (UniqueName: \"kubernetes.io/projected/3363df8b-023e-4b28-a595-8dd9f056f9da-kube-api-access-hkmh9\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.094528 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.094445 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.195533 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195499 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-sys\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.195533 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195535 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-textfile\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3363df8b-023e-4b28-a595-8dd9f056f9da-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-metrics-client-ca\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195696 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195714 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b65fc1a8-c522-453e-82e5-8d1997135aba-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195735 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-tls\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.195783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195771 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvzrf\" (UniqueName: \"kubernetes.io/projected/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-kube-api-access-zvzrf\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195822 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkmh9\" (UniqueName: \"kubernetes.io/projected/3363df8b-023e-4b28-a595-8dd9f056f9da-kube-api-access-hkmh9\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-root\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195881 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.195903 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:34.195985 2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.196025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws2w7\" (UniqueName: \"kubernetes.io/projected/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-api-access-ws2w7\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:34.196051 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-tls podName:3363df8b-023e-4b28-a595-8dd9f056f9da nodeName:}" failed. No retries permitted until 2026-09-10 04:42:34.69602973 +0000 UTC m=+80.791496449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-bzjn2" (UID: "3363df8b-023e-4b28-a595-8dd9f056f9da") : secret "openshift-state-metrics-tls" not found Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.196094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b65fc1a8-c522-453e-82e5-8d1997135aba-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.196139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.196123 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-wtmp\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.196483 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.196244 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3363df8b-023e-4b28-a595-8dd9f056f9da-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.198171 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.198153 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.215373 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.215348 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkmh9\" (UniqueName: \"kubernetes.io/projected/3363df8b-023e-4b28-a595-8dd9f056f9da-kube-api-access-hkmh9\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.296498 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296421 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-root\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296498 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296498 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296476 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws2w7\" (UniqueName: \"kubernetes.io/projected/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-api-access-ws2w7\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296502 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b65fc1a8-c522-453e-82e5-8d1997135aba-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-wtmp\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296543 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-root\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296670 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-wtmp\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296703 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-sys\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-textfile\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296781 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-sys\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.296783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296784 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-metrics-client-ca\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b65fc1a8-c522-453e-82e5-8d1997135aba-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296936 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-tls\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296959 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzrf\" (UniqueName: \"kubernetes.io/projected/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-kube-api-access-zvzrf\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.296992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.297172 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.297070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-textfile\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.297561 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.297350 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b65fc1a8-c522-453e-82e5-8d1997135aba-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.297561 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.297385 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.297561 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:34.297440 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 04:42:34.297561 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:42:34.297513 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-tls podName:aed15e65-8bce-4e3e-a05c-b1c1e116bbc0 nodeName:}" failed. No retries permitted until 2026-09-10 04:42:34.797495492 +0000 UTC m=+80.892962207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-tls") pod "node-exporter-s9vs6" (UID: "aed15e65-8bce-4e3e-a05c-b1c1e116bbc0") : secret "node-exporter-tls" not found Sep 10 04:42:34.297561 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.297532 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.297561 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.297555 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b65fc1a8-c522-453e-82e5-8d1997135aba-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.298229 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.298199 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-metrics-client-ca\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.299287 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.299267 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.299466 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.299449 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.299763 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.299747 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.308195 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.308177 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws2w7\" (UniqueName: \"kubernetes.io/projected/b65fc1a8-c522-453e-82e5-8d1997135aba-kube-api-access-ws2w7\") pod \"kube-state-metrics-74cb57df7-hqwnv\" (UID: \"b65fc1a8-c522-453e-82e5-8d1997135aba\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.308268 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.308191 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzrf\" (UniqueName: \"kubernetes.io/projected/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-kube-api-access-zvzrf\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.344748 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.344727 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" Sep 10 04:42:34.614513 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.614429 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv"] Sep 10 04:42:34.617164 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:34.617130 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb65fc1a8_c522_453e_82e5_8d1997135aba.slice/crio-0f3434a5a011d2d5b6de596d58e41f69561531de03b47f7ba8227946680a2cf7 WatchSource:0}: Error finding container 0f3434a5a011d2d5b6de596d58e41f69561531de03b47f7ba8227946680a2cf7: Status 404 returned error can't find the container with id 0f3434a5a011d2d5b6de596d58e41f69561531de03b47f7ba8227946680a2cf7 Sep 10 04:42:34.700466 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.700434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.702741 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.702722 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3363df8b-023e-4b28-a595-8dd9f056f9da-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-bzjn2\" (UID: \"3363df8b-023e-4b28-a595-8dd9f056f9da\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.801053 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.801022 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-tls\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.803247 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.803223 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed15e65-8bce-4e3e-a05c-b1c1e116bbc0-node-exporter-tls\") pod \"node-exporter-s9vs6\" (UID: \"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0\") " pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.855223 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.855190 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" event={"ID":"b65fc1a8-c522-453e-82e5-8d1997135aba","Type":"ContainerStarted","Data":"0f3434a5a011d2d5b6de596d58e41f69561531de03b47f7ba8227946680a2cf7"} Sep 10 04:42:34.911522 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.911461 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" Sep 10 04:42:34.949492 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:34.949460 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s9vs6" Sep 10 04:42:34.962737 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:34.962647 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed15e65_8bce_4e3e_a05c_b1c1e116bbc0.slice/crio-a99c54cd82e1f0e5bf8f4b2acd10e2e9e24c516f2b4a6f7f3cf9c11abb909a3c WatchSource:0}: Error finding container a99c54cd82e1f0e5bf8f4b2acd10e2e9e24c516f2b4a6f7f3cf9c11abb909a3c: Status 404 returned error can't find the container with id a99c54cd82e1f0e5bf8f4b2acd10e2e9e24c516f2b4a6f7f3cf9c11abb909a3c Sep 10 04:42:35.127211 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.127178 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 04:42:35.131993 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.131299 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.135463 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.135426 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 04:42:35.135581 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.135487 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 04:42:35.135679 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.135584 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-d6fmx\"" Sep 10 04:42:35.135797 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.135744 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 04:42:35.136075 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.136053 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 04:42:35.136215 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.136196 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 04:42:35.136297 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.136226 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 04:42:35.136507 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.136490 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 04:42:35.136775 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.136749 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 04:42:35.136870 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.136819 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 04:42:35.156333 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.156200 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 04:42:35.187912 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.187843 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2"] Sep 10 04:42:35.190157 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:35.190133 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3363df8b_023e_4b28_a595_8dd9f056f9da.slice/crio-54d08ad20526289d63c90f532e87d1263c18018d173cd5ce77b36c89c29505a7 WatchSource:0}: Error finding container 54d08ad20526289d63c90f532e87d1263c18018d173cd5ce77b36c89c29505a7: Status 404 returned error can't find the container with id 54d08ad20526289d63c90f532e87d1263c18018d173cd5ce77b36c89c29505a7 Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304539 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304678 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-config-volume\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304777 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chrs5\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-kube-api-access-chrs5\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-web-config\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304834 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304909 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-tls-assets\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.304975 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.305004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-config-out\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.305079 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.305035 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406019 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.405963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406019 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406267 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406047 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-config-volume\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406267 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406211 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406267 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chrs5\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-kube-api-access-chrs5\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406404 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-web-config\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406404 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406320 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406404 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406522 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406430 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406522 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-tls-assets\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406653 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406653 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406762 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406654 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-config-out\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.406762 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.406688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.407125 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.407098 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.407442 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.407417 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.409835 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.409806 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.410666 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.410621 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.411242 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.411157 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.411242 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.411175 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-tls-assets\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.411879 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.411827 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-config-volume\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.412162 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.412138 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-config-out\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.412814 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.412759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.414135 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.414111 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-web-config\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.414472 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.414293 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.416562 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.416540 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chrs5\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-kube-api-access-chrs5\") pod \"alertmanager-main-0\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.443715 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.443663 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:42:35.861201 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.861115 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" event={"ID":"3363df8b-023e-4b28-a595-8dd9f056f9da","Type":"ContainerStarted","Data":"f80340610e5b16571c8529526614c81015bddc0167420ec71442230d3bd322aa"} Sep 10 04:42:35.861201 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.861190 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" event={"ID":"3363df8b-023e-4b28-a595-8dd9f056f9da","Type":"ContainerStarted","Data":"d7ba7407572cab6a07687d75ea77652b4d48f3183cf0f361e876f71a4040ad12"} Sep 10 04:42:35.861411 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.861222 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" event={"ID":"3363df8b-023e-4b28-a595-8dd9f056f9da","Type":"ContainerStarted","Data":"54d08ad20526289d63c90f532e87d1263c18018d173cd5ce77b36c89c29505a7"} Sep 10 04:42:35.862487 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:35.862457 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s9vs6" event={"ID":"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0","Type":"ContainerStarted","Data":"a99c54cd82e1f0e5bf8f4b2acd10e2e9e24c516f2b4a6f7f3cf9c11abb909a3c"} Sep 10 04:42:36.287452 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.287362 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 04:42:36.289687 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:36.289647 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f88529_51cd_4e43_8a11_cd6830959131.slice/crio-1d7fb57a0c87e64a3a30309212dc5c1a791e97c31faca17f2549195a1b7a908e WatchSource:0}: Error finding container 1d7fb57a0c87e64a3a30309212dc5c1a791e97c31faca17f2549195a1b7a908e: Status 404 returned error can't find the container with id 1d7fb57a0c87e64a3a30309212dc5c1a791e97c31faca17f2549195a1b7a908e Sep 10 04:42:36.870137 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.870098 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" event={"ID":"b65fc1a8-c522-453e-82e5-8d1997135aba","Type":"ContainerStarted","Data":"600bc3f6de27e691550e240cc35caca00e4b6e6b8bad54858d7bd90c6409e13c"} Sep 10 04:42:36.870302 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.870161 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" event={"ID":"b65fc1a8-c522-453e-82e5-8d1997135aba","Type":"ContainerStarted","Data":"b9ca6d02c115b92a32f7a121efeca3b16b772ad464d96b0ba91409e5f607fcfd"} Sep 10 04:42:36.870302 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.870188 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" event={"ID":"b65fc1a8-c522-453e-82e5-8d1997135aba","Type":"ContainerStarted","Data":"acfa1206e24cf43f643727fee295a008146bcdb3927c5d3e3a22d8dcb282a60f"} Sep 10 04:42:36.873098 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.873063 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" event={"ID":"3363df8b-023e-4b28-a595-8dd9f056f9da","Type":"ContainerStarted","Data":"854aad62614ce6762d40eb0e2721b6954b8ceb0103f7762e2ada1e068593f03b"} Sep 10 04:42:36.875121 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.875093 2565 generic.go:356] "Generic (PLEG): container finished" podID="aed15e65-8bce-4e3e-a05c-b1c1e116bbc0" containerID="7d11180de55007d353bf381862d60d02e2c21cad81b7283bdf18d4b2df325ea0" exitCode=0 Sep 10 04:42:36.875248 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.875188 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s9vs6" event={"ID":"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0","Type":"ContainerDied","Data":"7d11180de55007d353bf381862d60d02e2c21cad81b7283bdf18d4b2df325ea0"} Sep 10 04:42:36.876501 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.876479 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerStarted","Data":"1d7fb57a0c87e64a3a30309212dc5c1a791e97c31faca17f2549195a1b7a908e"} Sep 10 04:42:36.898960 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.898911 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqwnv" podStartSLOduration=1.478586411 podStartE2EDuration="2.898892652s" totalImagesPullingTime="1.420306241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:36.898404655 +0000 UTC m=+82.993871416" watchObservedRunningTime="2026-09-10 04:42:36.898892652 +0000 UTC m=+82.994359390" Sep 10 04:42:36.948652 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:36.948588 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-bzjn2" podStartSLOduration=2.611154445 podStartE2EDuration="3.948572258s" totalImagesPullingTime="1.337417813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:36.922541359 +0000 UTC m=+83.018008120" watchObservedRunningTime="2026-09-10 04:42:36.948572258 +0000 UTC m=+83.044038996" Sep 10 04:42:37.835454 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:37.835418 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w82t5" Sep 10 04:42:37.883378 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:37.883347 2565 generic.go:356] "Generic (PLEG): container finished" podID="88f88529-51cd-4e43-8a11-cd6830959131" containerID="2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced" exitCode=0 Sep 10 04:42:37.883534 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:37.883434 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced"} Sep 10 04:42:37.886120 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:37.886089 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s9vs6" event={"ID":"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0","Type":"ContainerStarted","Data":"543ad14205c054def424e3c614bed29e32d8e5a95d3f7db573da536d6674e71b"} Sep 10 04:42:37.886214 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:37.886143 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s9vs6" event={"ID":"aed15e65-8bce-4e3e-a05c-b1c1e116bbc0","Type":"ContainerStarted","Data":"afc039a018d6bb1ba339ee62833d020bfcbe1438cfb1732a045a770b115867c5"} Sep 10 04:42:37.937332 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:37.937285 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s9vs6" podStartSLOduration=2.76764486 podStartE2EDuration="3.937268838s" totalImagesPullingTime="1.076247564s" totalInitContainerRuntime="93.376414ms" isStatefulPod=true podCreationTimestamp="2026-09-10 04:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:37.936367812 +0000 UTC m=+84.031834568" watchObservedRunningTime="2026-09-10 04:42:37.937268838 +0000 UTC m=+84.032735577" Sep 10 04:42:38.539368 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.539332 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fcffb998d-69rvp"] Sep 10 04:42:38.542641 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.542599 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.546336 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.546305 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 10 04:42:38.546452 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.546365 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 10 04:42:38.546452 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.546366 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 10 04:42:38.546723 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.546700 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-21lqma74laeeg\"" Sep 10 04:42:38.546827 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.546727 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 04:42:38.546827 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.546729 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2v6nl\"" Sep 10 04:42:38.553107 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.553053 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fcffb998d-69rvp"] Sep 10 04:42:38.635379 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.635348 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2kzh\" (UniqueName: \"kubernetes.io/projected/5330794d-4870-436c-8763-03237c61ffeb-kube-api-access-p2kzh\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.635579 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.635391 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5330794d-4870-436c-8763-03237c61ffeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.635579 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.635415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5330794d-4870-436c-8763-03237c61ffeb-audit-log\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.635579 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.635433 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-secret-metrics-server-client-certs\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.635579 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.635501 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-client-ca-bundle\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.635579 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.635554 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-secret-metrics-server-tls\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.635579 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.635575 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5330794d-4870-436c-8763-03237c61ffeb-metrics-server-audit-profiles\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736140 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2kzh\" (UniqueName: \"kubernetes.io/projected/5330794d-4870-436c-8763-03237c61ffeb-kube-api-access-p2kzh\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736140 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736146 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5330794d-4870-436c-8763-03237c61ffeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736335 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5330794d-4870-436c-8763-03237c61ffeb-audit-log\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736335 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736274 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-secret-metrics-server-client-certs\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736335 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-client-ca-bundle\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736495 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736411 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-secret-metrics-server-tls\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736495 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5330794d-4870-436c-8763-03237c61ffeb-metrics-server-audit-profiles\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736625 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736542 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5330794d-4870-436c-8763-03237c61ffeb-audit-log\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.736995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.736966 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5330794d-4870-436c-8763-03237c61ffeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.737395 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.737367 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5330794d-4870-436c-8763-03237c61ffeb-metrics-server-audit-profiles\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.738929 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.738908 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-client-ca-bundle\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.739068 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.739045 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-secret-metrics-server-tls\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.739248 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.739217 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5330794d-4870-436c-8763-03237c61ffeb-secret-metrics-server-client-certs\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.747394 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.747368 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2kzh\" (UniqueName: \"kubernetes.io/projected/5330794d-4870-436c-8763-03237c61ffeb-kube-api-access-p2kzh\") pod \"metrics-server-6fcffb998d-69rvp\" (UID: \"5330794d-4870-436c-8763-03237c61ffeb\") " pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:38.854790 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:38.854712 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:39.135578 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:39.135188 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fcffb998d-69rvp"] Sep 10 04:42:39.413101 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:39.413034 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5330794d_4870_436c_8763_03237c61ffeb.slice/crio-92606fb05168f39958cf629503cb123ba5a7ec848ca6189544e57781a3502d08 WatchSource:0}: Error finding container 92606fb05168f39958cf629503cb123ba5a7ec848ca6189544e57781a3502d08: Status 404 returned error can't find the container with id 92606fb05168f39958cf629503cb123ba5a7ec848ca6189544e57781a3502d08 Sep 10 04:42:39.898402 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:39.898368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerStarted","Data":"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53"} Sep 10 04:42:39.898825 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:39.898442 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerStarted","Data":"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862"} Sep 10 04:42:39.898825 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:39.898479 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerStarted","Data":"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499"} Sep 10 04:42:39.898825 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:39.898506 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerStarted","Data":"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4"} Sep 10 04:42:39.898825 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:39.898529 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerStarted","Data":"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f"} Sep 10 04:42:39.899555 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:39.899531 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" event={"ID":"5330794d-4870-436c-8763-03237c61ffeb","Type":"ContainerStarted","Data":"92606fb05168f39958cf629503cb123ba5a7ec848ca6189544e57781a3502d08"} Sep 10 04:42:40.336029 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.335993 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:42:40.340341 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.340318 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.345371 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.345334 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 04:42:40.345371 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.345360 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 04:42:40.346826 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.346803 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 04:42:40.347056 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347038 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 04:42:40.347181 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347163 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dj0g9ulqr16j5\"" Sep 10 04:42:40.347181 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347175 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 04:42:40.347302 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347215 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 04:42:40.347302 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347228 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 04:42:40.347302 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347164 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 04:42:40.347438 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347356 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 04:42:40.347438 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347405 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 04:42:40.347438 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.347177 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 04:42:40.348048 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.348029 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bkjk4\"" Sep 10 04:42:40.348838 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.348816 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 04:42:40.349478 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.349458 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 04:42:40.355338 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.355307 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 04:42:40.365964 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.365937 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451235 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b59d\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-kube-api-access-7b59d\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451265 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451317 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451362 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451393 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451427 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451513 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451539 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-config-out\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451572 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-web-config\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451596 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.451995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-config\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.452926 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451704 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.452926 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451749 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.452926 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451775 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.452926 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.451803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553011 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.552979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553021 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553043 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b59d\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-kube-api-access-7b59d\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553101 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553116 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553132 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553146 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553163 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553178 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553203 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553234 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553276 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-config-out\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553298 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-web-config\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553311 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553329 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553349 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-config\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.553721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.553370 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.554174 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.554039 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.554275 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.554251 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.554480 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.554454 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.555212 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.554917 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.555212 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.554919 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.555212 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.555128 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.558439 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.558092 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.558439 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.558159 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.559857 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.559826 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.560469 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.560427 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-config-out\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.561275 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.560856 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-config\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.561275 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.560955 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.562147 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.562103 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.562147 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.562128 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.562906 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.562883 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.563048 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.562993 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.563422 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.563403 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.563970 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.563949 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.564273 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.564250 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-web-config\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.565468 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.565404 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b59d\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-kube-api-access-7b59d\") pod \"prometheus-k8s-0\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:40.653915 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:40.653825 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:41.300373 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.300347 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:42:41.302005 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:42:41.301983 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5846bd9_7fe7_497a_95d8_798518b02028.slice/crio-ca9a555193621aa47410d91653db2ecf90bdadac770bb20b99657300f5570ba9 WatchSource:0}: Error finding container ca9a555193621aa47410d91653db2ecf90bdadac770bb20b99657300f5570ba9: Status 404 returned error can't find the container with id ca9a555193621aa47410d91653db2ecf90bdadac770bb20b99657300f5570ba9 Sep 10 04:42:41.797258 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.797229 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56775dcb96-hkm7j" Sep 10 04:42:41.909823 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.909784 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" event={"ID":"5330794d-4870-436c-8763-03237c61ffeb","Type":"ContainerStarted","Data":"6ac36fa4bd706b9475a6c9f0a025c72930d6c6f093b11edb3b7b91ec138dbc7f"} Sep 10 04:42:41.911354 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.911329 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5846bd9-7fe7-497a-95d8-798518b02028" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" exitCode=0 Sep 10 04:42:41.911471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.911423 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a"} Sep 10 04:42:41.911527 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.911497 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerStarted","Data":"ca9a555193621aa47410d91653db2ecf90bdadac770bb20b99657300f5570ba9"} Sep 10 04:42:41.915094 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.915075 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerStarted","Data":"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2"} Sep 10 04:42:41.932668 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.932625 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" podStartSLOduration=2.326693494 podStartE2EDuration="3.932588868s" totalImagesPullingTime="1.605895374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:41.931435875 +0000 UTC m=+88.026902612" watchObservedRunningTime="2026-09-10 04:42:41.932588868 +0000 UTC m=+88.028055605" Sep 10 04:42:41.971150 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:41.971105 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.103949585 podStartE2EDuration="6.971088507s" totalImagesPullingTime="3.848851895s" totalInitContainerRuntime="18.287027ms" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:41.969407136 +0000 UTC m=+88.064873894" watchObservedRunningTime="2026-09-10 04:42:41.971088507 +0000 UTC m=+88.066555242" Sep 10 04:42:45.939777 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:45.939739 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerStarted","Data":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} Sep 10 04:42:45.940160 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:45.939852 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerStarted","Data":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} Sep 10 04:42:47.952464 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:47.952428 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerStarted","Data":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} Sep 10 04:42:47.952854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:47.952509 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerStarted","Data":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} Sep 10 04:42:47.952854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:47.952548 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerStarted","Data":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} Sep 10 04:42:47.952854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:47.952587 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerStarted","Data":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} Sep 10 04:42:47.988589 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:47.988529 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.922127991 podStartE2EDuration="7.988510164s" totalImagesPullingTime="5.04228943s" totalInitContainerRuntime="24.092743ms" isStatefulPod=false podCreationTimestamp="2026-09-10 04:42:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:42:47.985772231 +0000 UTC m=+94.081238987" watchObservedRunningTime="2026-09-10 04:42:47.988510164 +0000 UTC m=+94.083976904" Sep 10 04:42:50.654694 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:50.654648 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:42:58.834714 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:58.834683 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8pzhh" Sep 10 04:42:58.855293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:58.855259 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:42:58.859622 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:42:58.855623 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:43:18.860048 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:18.860020 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:43:18.863741 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:18.863719 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fcffb998d-69rvp" Sep 10 04:43:40.654790 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:40.654752 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:40.673445 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:40.673417 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:41.168103 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:41.168074 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:54.450984 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:54.450944 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 04:43:54.451512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:54.451329 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="alertmanager" containerID="cri-o://782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f" gracePeriod=120 Sep 10 04:43:54.451512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:54.451369 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-metric" containerID="cri-o://c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53" gracePeriod=120 Sep 10 04:43:54.451512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:54.451403 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-web" containerID="cri-o://5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499" gracePeriod=120 Sep 10 04:43:54.451512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:54.451417 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="config-reloader" containerID="cri-o://f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4" gracePeriod=120 Sep 10 04:43:54.451512 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:54.451447 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="prom-label-proxy" containerID="cri-o://030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2" gracePeriod=120 Sep 10 04:43:54.451731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:54.451589 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy" containerID="cri-o://18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862" gracePeriod=120 Sep 10 04:43:55.206232 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206197 2565 generic.go:356] "Generic (PLEG): container finished" podID="88f88529-51cd-4e43-8a11-cd6830959131" containerID="030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2" exitCode=0 Sep 10 04:43:55.206232 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206222 2565 generic.go:356] "Generic (PLEG): container finished" podID="88f88529-51cd-4e43-8a11-cd6830959131" containerID="18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862" exitCode=0 Sep 10 04:43:55.206232 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206228 2565 generic.go:356] "Generic (PLEG): container finished" podID="88f88529-51cd-4e43-8a11-cd6830959131" containerID="f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4" exitCode=0 Sep 10 04:43:55.206232 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206234 2565 generic.go:356] "Generic (PLEG): container finished" podID="88f88529-51cd-4e43-8a11-cd6830959131" containerID="782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f" exitCode=0 Sep 10 04:43:55.206495 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206255 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2"} Sep 10 04:43:55.206495 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206325 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862"} Sep 10 04:43:55.206495 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206352 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4"} Sep 10 04:43:55.206495 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.206376 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f"} Sep 10 04:43:55.688786 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.688762 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:43:55.763097 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763025 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-main-db\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763097 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763081 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-main-tls\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763299 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763104 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-config-out\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763299 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763130 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-tls-assets\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763299 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763166 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763299 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763182 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-cluster-tls-config\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763299 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763229 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-web-config\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763299 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763257 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chrs5\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-kube-api-access-chrs5\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763303 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-metrics-client-ca\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763332 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-config-volume\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763357 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763383 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-web\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.763689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763389 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 04:43:55.763689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763421 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-trusted-ca-bundle\") pod \"88f88529-51cd-4e43-8a11-cd6830959131\" (UID: \"88f88529-51cd-4e43-8a11-cd6830959131\") " Sep 10 04:43:55.764003 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763702 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-main-db\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.764003 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.763986 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:55.764100 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.764067 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:55.766362 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.766335 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:43:55.766475 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.766358 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-config-out" (OuterVolumeSpecName: "config-out") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 04:43:55.766673 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.766653 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:55.766897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.766864 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-kube-api-access-chrs5" (OuterVolumeSpecName: "kube-api-access-chrs5") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "kube-api-access-chrs5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:43:55.766976 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.766910 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:55.767086 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.767066 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:55.768428 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.768402 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:55.768428 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.768412 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-config-volume" (OuterVolumeSpecName: "config-volume") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:55.772210 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.772184 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:55.778303 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.778282 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-web-config" (OuterVolumeSpecName: "web-config") pod "88f88529-51cd-4e43-8a11-cd6830959131" (UID: "88f88529-51cd-4e43-8a11-cd6830959131"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:55.864752 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864729 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864752 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864751 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-cluster-tls-config\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864762 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-web-config\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864772 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chrs5\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-kube-api-access-chrs5\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864780 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-metrics-client-ca\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864788 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-config-volume\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864802 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864811 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864819 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f88529-51cd-4e43-8a11-cd6830959131-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864829 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/88f88529-51cd-4e43-8a11-cd6830959131-secret-alertmanager-main-tls\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864836 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88f88529-51cd-4e43-8a11-cd6830959131-config-out\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:55.864875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:55.864844 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88f88529-51cd-4e43-8a11-cd6830959131-tls-assets\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:56.213522 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.213494 2565 generic.go:356] "Generic (PLEG): container finished" podID="88f88529-51cd-4e43-8a11-cd6830959131" containerID="c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53" exitCode=0 Sep 10 04:43:56.213522 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.213519 2565 generic.go:356] "Generic (PLEG): container finished" podID="88f88529-51cd-4e43-8a11-cd6830959131" containerID="5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499" exitCode=0 Sep 10 04:43:56.213731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.213583 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 04:43:56.213731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.213571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53"} Sep 10 04:43:56.213797 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.213782 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499"} Sep 10 04:43:56.213842 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.213830 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"88f88529-51cd-4e43-8a11-cd6830959131","Type":"ContainerDied","Data":"1d7fb57a0c87e64a3a30309212dc5c1a791e97c31faca17f2549195a1b7a908e"} Sep 10 04:43:56.213876 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.213855 2565 scope.go:117] "RemoveContainer" containerID="030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2" Sep 10 04:43:56.222995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.222977 2565 scope.go:117] "RemoveContainer" containerID="c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53" Sep 10 04:43:56.229961 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.229946 2565 scope.go:117] "RemoveContainer" containerID="18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862" Sep 10 04:43:56.236619 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.236588 2565 scope.go:117] "RemoveContainer" containerID="5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499" Sep 10 04:43:56.238066 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.238047 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 04:43:56.242208 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.242189 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 04:43:56.244552 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.244536 2565 scope.go:117] "RemoveContainer" containerID="f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4" Sep 10 04:43:56.251111 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.251092 2565 scope.go:117] "RemoveContainer" containerID="782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f" Sep 10 04:43:56.258075 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.258056 2565 scope.go:117] "RemoveContainer" containerID="2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced" Sep 10 04:43:56.264733 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.264715 2565 scope.go:117] "RemoveContainer" containerID="030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2" Sep 10 04:43:56.265005 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:56.264985 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2\": container with ID starting with 030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2 not found: ID does not exist" containerID="030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2" Sep 10 04:43:56.265084 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.265013 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2"} err="failed to get container status \"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2\": rpc error: code = NotFound desc = could not find container \"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2\": container with ID starting with 030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2 not found: ID does not exist" Sep 10 04:43:56.265084 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.265044 2565 scope.go:117] "RemoveContainer" containerID="c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53" Sep 10 04:43:56.265376 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:56.265321 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53\": container with ID starting with c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53 not found: ID does not exist" containerID="c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53" Sep 10 04:43:56.265470 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.265393 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53"} err="failed to get container status \"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53\": rpc error: code = NotFound desc = could not find container \"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53\": container with ID starting with c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53 not found: ID does not exist" Sep 10 04:43:56.265553 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.265471 2565 scope.go:117] "RemoveContainer" containerID="18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862" Sep 10 04:43:56.265979 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:56.265960 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862\": container with ID starting with 18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862 not found: ID does not exist" containerID="18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862" Sep 10 04:43:56.266054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.265983 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862"} err="failed to get container status \"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862\": rpc error: code = NotFound desc = could not find container \"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862\": container with ID starting with 18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862 not found: ID does not exist" Sep 10 04:43:56.266054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.265999 2565 scope.go:117] "RemoveContainer" containerID="5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499" Sep 10 04:43:56.266263 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:56.266246 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499\": container with ID starting with 5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499 not found: ID does not exist" containerID="5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499" Sep 10 04:43:56.266298 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.266267 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499"} err="failed to get container status \"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499\": rpc error: code = NotFound desc = could not find container \"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499\": container with ID starting with 5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499 not found: ID does not exist" Sep 10 04:43:56.266298 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.266280 2565 scope.go:117] "RemoveContainer" containerID="f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4" Sep 10 04:43:56.266533 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:56.266514 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4\": container with ID starting with f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4 not found: ID does not exist" containerID="f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4" Sep 10 04:43:56.266572 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.266538 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4"} err="failed to get container status \"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4\": rpc error: code = NotFound desc = could not find container \"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4\": container with ID starting with f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4 not found: ID does not exist" Sep 10 04:43:56.266572 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.266553 2565 scope.go:117] "RemoveContainer" containerID="782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f" Sep 10 04:43:56.266859 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:56.266840 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f\": container with ID starting with 782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f not found: ID does not exist" containerID="782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f" Sep 10 04:43:56.266922 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.266873 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f"} err="failed to get container status \"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f\": rpc error: code = NotFound desc = could not find container \"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f\": container with ID starting with 782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f not found: ID does not exist" Sep 10 04:43:56.266922 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.266889 2565 scope.go:117] "RemoveContainer" containerID="2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced" Sep 10 04:43:56.267144 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:56.267126 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced\": container with ID starting with 2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced not found: ID does not exist" containerID="2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced" Sep 10 04:43:56.267187 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267149 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced"} err="failed to get container status \"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced\": rpc error: code = NotFound desc = could not find container \"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced\": container with ID starting with 2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced not found: ID does not exist" Sep 10 04:43:56.267187 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267162 2565 scope.go:117] "RemoveContainer" containerID="030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2" Sep 10 04:43:56.267388 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267370 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2"} err="failed to get container status \"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2\": rpc error: code = NotFound desc = could not find container \"030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2\": container with ID starting with 030aa4a7a4276bbc2861acf2d1515a65847a0ec8316d039ac78da84d92617fd2 not found: ID does not exist" Sep 10 04:43:56.267434 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267389 2565 scope.go:117] "RemoveContainer" containerID="c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53" Sep 10 04:43:56.267622 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267583 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53"} err="failed to get container status \"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53\": rpc error: code = NotFound desc = could not find container \"c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53\": container with ID starting with c4af1e1c9a43289f533d816790fd842b0e1b6e98ca4982d544ac37a89df47f53 not found: ID does not exist" Sep 10 04:43:56.267694 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267628 2565 scope.go:117] "RemoveContainer" containerID="18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862" Sep 10 04:43:56.267868 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267846 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862"} err="failed to get container status \"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862\": rpc error: code = NotFound desc = could not find container \"18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862\": container with ID starting with 18c06538df954b0cabc8c946942a5b5915c3ae617112aeadb0d8d152bad42862 not found: ID does not exist" Sep 10 04:43:56.267909 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.267872 2565 scope.go:117] "RemoveContainer" containerID="5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499" Sep 10 04:43:56.268114 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.268095 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499"} err="failed to get container status \"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499\": rpc error: code = NotFound desc = could not find container \"5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499\": container with ID starting with 5252544cf8052256c617b356585cd0c13c0eff24e16b47c9eb59d3697954c499 not found: ID does not exist" Sep 10 04:43:56.268175 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.268116 2565 scope.go:117] "RemoveContainer" containerID="f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4" Sep 10 04:43:56.268473 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.268452 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4"} err="failed to get container status \"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4\": rpc error: code = NotFound desc = could not find container \"f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4\": container with ID starting with f4d8fb503ab91b980fe32b90312b8b6bd40d7f78907b82109489eac880935ed4 not found: ID does not exist" Sep 10 04:43:56.268549 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.268473 2565 scope.go:117] "RemoveContainer" containerID="782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f" Sep 10 04:43:56.268759 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.268740 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f"} err="failed to get container status \"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f\": rpc error: code = NotFound desc = could not find container \"782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f\": container with ID starting with 782a829f3a13ee728bbf2b1edd8fc0a892bd12c0214e1525fd853f0ff04a259f not found: ID does not exist" Sep 10 04:43:56.268759 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.268758 2565 scope.go:117] "RemoveContainer" containerID="2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced" Sep 10 04:43:56.269009 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.268989 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced"} err="failed to get container status \"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced\": rpc error: code = NotFound desc = could not find container \"2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced\": container with ID starting with 2fc20552847f71a585e996d8f0003e6f531269acdbf48551f3f42cc7e8009ced not found: ID does not exist" Sep 10 04:43:56.383683 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:56.383655 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88f88529-51cd-4e43-8a11-cd6830959131" path="/var/lib/kubelet/pods/88f88529-51cd-4e43-8a11-cd6830959131/volumes" Sep 10 04:43:58.668837 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.668804 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:43:58.669279 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.669185 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="prometheus" containerID="cri-o://d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" gracePeriod=600 Sep 10 04:43:58.669385 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.669361 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-web" containerID="cri-o://3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" gracePeriod=600 Sep 10 04:43:58.669457 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.669369 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy" containerID="cri-o://276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" gracePeriod=600 Sep 10 04:43:58.669516 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.669470 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="thanos-sidecar" containerID="cri-o://c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" gracePeriod=600 Sep 10 04:43:58.669516 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.669464 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" gracePeriod=600 Sep 10 04:43:58.669637 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.669462 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="config-reloader" containerID="cri-o://c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" gracePeriod=600 Sep 10 04:43:58.919665 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.919587 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:58.988311 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988278 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-trusted-ca-bundle\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988326 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-metrics-client-ca\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988353 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-tls\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988376 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-config-out\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988405 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-0\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988451 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988442 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-1\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988717 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:58.988898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988724 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:58.988898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988777 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-kubelet-serving-ca-bundle\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988816 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-web-config\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988858 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-serving-certs-ca-bundle\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.988898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988885 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-tls-assets\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988914 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b59d\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-kube-api-access-7b59d\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988940 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-2\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988997 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.988999 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989024 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-metrics-client-certs\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989058 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-db\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989096 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989118 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-kube-rbac-proxy\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989152 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-thanos-prometheus-http-client-file\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989194 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-grpc-tls\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989220 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-config\") pod \"d5846bd9-7fe7-497a-95d8-798518b02028\" (UID: \"d5846bd9-7fe7-497a-95d8-798518b02028\") " Sep 10 04:43:58.989786 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989392 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:58.989786 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989505 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:58.989786 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989524 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-metrics-client-ca\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:58.989786 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989540 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:58.989786 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989555 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:58.990033 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.989826 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:58.990204 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.990178 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:58.991775 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.991464 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 04:43:58.992170 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.992142 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.992272 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.992215 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.992976 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.992946 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 04:43:58.993812 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.993775 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.994052 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994019 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:43:58.994226 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994193 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-config" (OuterVolumeSpecName: "config") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.994292 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994261 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.994372 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994352 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-config-out" (OuterVolumeSpecName: "config-out") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 04:43:58.994491 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994472 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.994624 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994573 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.994720 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994638 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:58.994720 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:58.994662 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-kube-api-access-7b59d" (OuterVolumeSpecName: "kube-api-access-7b59d") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "kube-api-access-7b59d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:43:59.003053 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.003028 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-web-config" (OuterVolumeSpecName: "web-config") pod "d5846bd9-7fe7-497a-95d8-798518b02028" (UID: "d5846bd9-7fe7-497a-95d8-798518b02028"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:43:59.090125 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090100 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-db\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090125 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090122 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090135 2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-kube-rbac-proxy\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090144 2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090153 2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-grpc-tls\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090161 2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-config\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090169 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090177 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5846bd9-7fe7-497a-95d8-798518b02028-config-out\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090186 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090194 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090203 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-web-config\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090212 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5846bd9-7fe7-497a-95d8-798518b02028-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090221 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-tls-assets\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090228 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b59d\" (UniqueName: \"kubernetes.io/projected/d5846bd9-7fe7-497a-95d8-798518b02028-kube-api-access-7b59d\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090236 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.090293 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.090246 2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d5846bd9-7fe7-497a-95d8-798518b02028-secret-metrics-client-certs\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:43:59.229435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229368 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5846bd9-7fe7-497a-95d8-798518b02028" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" exitCode=0 Sep 10 04:43:59.229435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229392 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5846bd9-7fe7-497a-95d8-798518b02028" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" exitCode=0 Sep 10 04:43:59.229435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229399 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5846bd9-7fe7-497a-95d8-798518b02028" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" exitCode=0 Sep 10 04:43:59.229435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229406 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5846bd9-7fe7-497a-95d8-798518b02028" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" exitCode=0 Sep 10 04:43:59.229435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229412 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5846bd9-7fe7-497a-95d8-798518b02028" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" exitCode=0 Sep 10 04:43:59.229435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229417 2565 generic.go:356] "Generic (PLEG): container finished" podID="d5846bd9-7fe7-497a-95d8-798518b02028" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" exitCode=0 Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229446 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229449 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229533 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229577 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229594 2565 scope.go:117] "RemoveContainer" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229634 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229677 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} Sep 10 04:43:59.229731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229718 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} Sep 10 04:43:59.230034 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.229760 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d5846bd9-7fe7-497a-95d8-798518b02028","Type":"ContainerDied","Data":"ca9a555193621aa47410d91653db2ecf90bdadac770bb20b99657300f5570ba9"} Sep 10 04:43:59.237681 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.237661 2565 scope.go:117] "RemoveContainer" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.248105 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.246358 2565 scope.go:117] "RemoveContainer" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.255060 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.255021 2565 scope.go:117] "RemoveContainer" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.257197 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.257176 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:43:59.260884 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.260865 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:43:59.263692 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.263645 2565 scope.go:117] "RemoveContainer" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.270174 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.270157 2565 scope.go:117] "RemoveContainer" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.276932 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.276915 2565 scope.go:117] "RemoveContainer" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.283094 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.283075 2565 scope.go:117] "RemoveContainer" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.283355 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:59.283337 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": container with ID starting with a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426 not found: ID does not exist" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.283414 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.283360 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} err="failed to get container status \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": rpc error: code = NotFound desc = could not find container \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": container with ID starting with a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426 not found: ID does not exist" Sep 10 04:43:59.283414 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.283377 2565 scope.go:117] "RemoveContainer" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.283641 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:59.283626 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": container with ID starting with 276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8 not found: ID does not exist" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.283695 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.283645 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} err="failed to get container status \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": rpc error: code = NotFound desc = could not find container \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": container with ID starting with 276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8 not found: ID does not exist" Sep 10 04:43:59.283695 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.283657 2565 scope.go:117] "RemoveContainer" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.283900 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:59.283884 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": container with ID starting with 3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129 not found: ID does not exist" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.283946 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.283906 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} err="failed to get container status \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": rpc error: code = NotFound desc = could not find container \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": container with ID starting with 3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129 not found: ID does not exist" Sep 10 04:43:59.283946 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.283922 2565 scope.go:117] "RemoveContainer" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.284177 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:59.284151 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": container with ID starting with c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718 not found: ID does not exist" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.284215 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284183 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} err="failed to get container status \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": rpc error: code = NotFound desc = could not find container \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": container with ID starting with c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718 not found: ID does not exist" Sep 10 04:43:59.284215 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284197 2565 scope.go:117] "RemoveContainer" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.284411 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:59.284397 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": container with ID starting with c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab not found: ID does not exist" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.284457 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284412 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} err="failed to get container status \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": rpc error: code = NotFound desc = could not find container \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": container with ID starting with c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab not found: ID does not exist" Sep 10 04:43:59.284457 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284424 2565 scope.go:117] "RemoveContainer" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.284654 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:59.284635 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": container with ID starting with d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43 not found: ID does not exist" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.284733 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284657 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} err="failed to get container status \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": rpc error: code = NotFound desc = could not find container \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": container with ID starting with d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43 not found: ID does not exist" Sep 10 04:43:59.284733 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284673 2565 scope.go:117] "RemoveContainer" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.284926 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:43:59.284909 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": container with ID starting with a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a not found: ID does not exist" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.284968 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284929 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a"} err="failed to get container status \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": rpc error: code = NotFound desc = could not find container \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": container with ID starting with a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a not found: ID does not exist" Sep 10 04:43:59.284968 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.284941 2565 scope.go:117] "RemoveContainer" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.285147 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285124 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} err="failed to get container status \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": rpc error: code = NotFound desc = could not find container \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": container with ID starting with a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426 not found: ID does not exist" Sep 10 04:43:59.285147 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285145 2565 scope.go:117] "RemoveContainer" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.285375 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285357 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} err="failed to get container status \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": rpc error: code = NotFound desc = could not find container \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": container with ID starting with 276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8 not found: ID does not exist" Sep 10 04:43:59.285413 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285375 2565 scope.go:117] "RemoveContainer" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.285627 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285594 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} err="failed to get container status \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": rpc error: code = NotFound desc = could not find container \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": container with ID starting with 3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129 not found: ID does not exist" Sep 10 04:43:59.285673 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285628 2565 scope.go:117] "RemoveContainer" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.285861 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285844 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} err="failed to get container status \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": rpc error: code = NotFound desc = could not find container \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": container with ID starting with c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718 not found: ID does not exist" Sep 10 04:43:59.285900 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.285862 2565 scope.go:117] "RemoveContainer" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.286054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286038 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} err="failed to get container status \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": rpc error: code = NotFound desc = could not find container \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": container with ID starting with c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab not found: ID does not exist" Sep 10 04:43:59.286094 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286055 2565 scope.go:117] "RemoveContainer" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.286239 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286225 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} err="failed to get container status \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": rpc error: code = NotFound desc = could not find container \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": container with ID starting with d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43 not found: ID does not exist" Sep 10 04:43:59.286276 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286239 2565 scope.go:117] "RemoveContainer" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.286426 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286411 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a"} err="failed to get container status \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": rpc error: code = NotFound desc = could not find container \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": container with ID starting with a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a not found: ID does not exist" Sep 10 04:43:59.286467 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286426 2565 scope.go:117] "RemoveContainer" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.286637 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286622 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} err="failed to get container status \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": rpc error: code = NotFound desc = could not find container \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": container with ID starting with a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426 not found: ID does not exist" Sep 10 04:43:59.286674 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286637 2565 scope.go:117] "RemoveContainer" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.286824 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286809 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} err="failed to get container status \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": rpc error: code = NotFound desc = could not find container \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": container with ID starting with 276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8 not found: ID does not exist" Sep 10 04:43:59.286864 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286824 2565 scope.go:117] "RemoveContainer" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.287001 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.286986 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} err="failed to get container status \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": rpc error: code = NotFound desc = could not find container \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": container with ID starting with 3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129 not found: ID does not exist" Sep 10 04:43:59.287041 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287001 2565 scope.go:117] "RemoveContainer" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.287184 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287169 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} err="failed to get container status \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": rpc error: code = NotFound desc = could not find container \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": container with ID starting with c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718 not found: ID does not exist" Sep 10 04:43:59.287220 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287184 2565 scope.go:117] "RemoveContainer" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.287362 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287347 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} err="failed to get container status \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": rpc error: code = NotFound desc = could not find container \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": container with ID starting with c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab not found: ID does not exist" Sep 10 04:43:59.287403 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287362 2565 scope.go:117] "RemoveContainer" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.287543 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287528 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} err="failed to get container status \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": rpc error: code = NotFound desc = could not find container \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": container with ID starting with d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43 not found: ID does not exist" Sep 10 04:43:59.287580 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287543 2565 scope.go:117] "RemoveContainer" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.287875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287831 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a"} err="failed to get container status \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": rpc error: code = NotFound desc = could not find container \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": container with ID starting with a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a not found: ID does not exist" Sep 10 04:43:59.287875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.287866 2565 scope.go:117] "RemoveContainer" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.288239 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.288201 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} err="failed to get container status \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": rpc error: code = NotFound desc = could not find container \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": container with ID starting with a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426 not found: ID does not exist" Sep 10 04:43:59.288239 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.288238 2565 scope.go:117] "RemoveContainer" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.288659 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.288636 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} err="failed to get container status \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": rpc error: code = NotFound desc = could not find container \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": container with ID starting with 276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8 not found: ID does not exist" Sep 10 04:43:59.288752 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.288714 2565 scope.go:117] "RemoveContainer" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.289040 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289005 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} err="failed to get container status \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": rpc error: code = NotFound desc = could not find container \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": container with ID starting with 3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129 not found: ID does not exist" Sep 10 04:43:59.289103 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289042 2565 scope.go:117] "RemoveContainer" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.289298 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289280 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} err="failed to get container status \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": rpc error: code = NotFound desc = could not find container \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": container with ID starting with c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718 not found: ID does not exist" Sep 10 04:43:59.289350 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289299 2565 scope.go:117] "RemoveContainer" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.289521 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289503 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} err="failed to get container status \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": rpc error: code = NotFound desc = could not find container \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": container with ID starting with c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab not found: ID does not exist" Sep 10 04:43:59.289580 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289522 2565 scope.go:117] "RemoveContainer" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.289580 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289555 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:43:59.289788 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289771 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} err="failed to get container status \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": rpc error: code = NotFound desc = could not find container \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": container with ID starting with d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43 not found: ID does not exist" Sep 10 04:43:59.289848 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289788 2565 scope.go:117] "RemoveContainer" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.289904 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289889 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-metric" Sep 10 04:43:59.289939 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289908 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-metric" Sep 10 04:43:59.289939 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289924 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="alertmanager" Sep 10 04:43:59.289939 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289933 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="alertmanager" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289939 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="config-reloader" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289945 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="config-reloader" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289956 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="thanos-sidecar" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289963 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="thanos-sidecar" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289969 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289974 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289979 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-thanos" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289984 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-thanos" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.289995 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-web" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290003 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-web" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290013 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="init-config-reloader" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290010 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a"} err="failed to get container status \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": rpc error: code = NotFound desc = could not find container \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": container with ID starting with a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a not found: ID does not exist" Sep 10 04:43:59.290024 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290028 2565 scope.go:117] "RemoveContainer" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290018 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="init-config-reloader" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290077 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290085 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290092 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="prom-label-proxy" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290097 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="prom-label-proxy" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290104 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="init-config-reloader" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290109 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="init-config-reloader" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290121 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="prometheus" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290126 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="prometheus" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290132 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="config-reloader" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290137 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="config-reloader" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290143 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-web" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290149 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-web" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290220 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290229 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-metric" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290235 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="config-reloader" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290242 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="thanos-sidecar" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290250 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-web" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290249 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} err="failed to get container status \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": rpc error: code = NotFound desc = could not find container \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": container with ID starting with a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426 not found: ID does not exist" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290259 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="alertmanager" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290264 2565 scope.go:117] "RemoveContainer" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290266 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="prometheus" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290312 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290319 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="kube-rbac-proxy-thanos" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290326 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="kube-rbac-proxy-web" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290333 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f88529-51cd-4e43-8a11-cd6830959131" containerName="prom-label-proxy" Sep 10 04:43:59.290471 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290342 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" containerName="config-reloader" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290492 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} err="failed to get container status \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": rpc error: code = NotFound desc = could not find container \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": container with ID starting with 276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8 not found: ID does not exist" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290513 2565 scope.go:117] "RemoveContainer" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290774 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} err="failed to get container status \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": rpc error: code = NotFound desc = could not find container \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": container with ID starting with 3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129 not found: ID does not exist" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.290792 2565 scope.go:117] "RemoveContainer" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.291139 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} err="failed to get container status \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": rpc error: code = NotFound desc = could not find container \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": container with ID starting with c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718 not found: ID does not exist" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.291162 2565 scope.go:117] "RemoveContainer" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.291570 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} err="failed to get container status \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": rpc error: code = NotFound desc = could not find container \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": container with ID starting with c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab not found: ID does not exist" Sep 10 04:43:59.291727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.291590 2565 scope.go:117] "RemoveContainer" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.292056 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.291873 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} err="failed to get container status \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": rpc error: code = NotFound desc = could not find container \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": container with ID starting with d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43 not found: ID does not exist" Sep 10 04:43:59.292056 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.291890 2565 scope.go:117] "RemoveContainer" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.292170 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292152 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a"} err="failed to get container status \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": rpc error: code = NotFound desc = could not find container \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": container with ID starting with a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a not found: ID does not exist" Sep 10 04:43:59.292205 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292171 2565 scope.go:117] "RemoveContainer" containerID="a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426" Sep 10 04:43:59.292422 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292402 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426"} err="failed to get container status \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": rpc error: code = NotFound desc = could not find container \"a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426\": container with ID starting with a090104ac3ad76ea981d3c787babc54355beabb5f79fd3f7d70a414fdb663426 not found: ID does not exist" Sep 10 04:43:59.292463 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292423 2565 scope.go:117] "RemoveContainer" containerID="276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8" Sep 10 04:43:59.292675 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292657 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8"} err="failed to get container status \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": rpc error: code = NotFound desc = could not find container \"276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8\": container with ID starting with 276af698b5af7a1106051d5fe7bee8c5ee5e1443e7fa29d1d6b45e06ab8ed6e8 not found: ID does not exist" Sep 10 04:43:59.292772 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292676 2565 scope.go:117] "RemoveContainer" containerID="3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129" Sep 10 04:43:59.292919 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292903 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129"} err="failed to get container status \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": rpc error: code = NotFound desc = could not find container \"3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129\": container with ID starting with 3da395004c900b9496fbd813fcd3d2e9b097747316747af8943603cd9848c129 not found: ID does not exist" Sep 10 04:43:59.292919 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.292918 2565 scope.go:117] "RemoveContainer" containerID="c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718" Sep 10 04:43:59.293115 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.293100 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718"} err="failed to get container status \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": rpc error: code = NotFound desc = could not find container \"c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718\": container with ID starting with c016e812669a376851ab5406937c31a322d83af5f00001c5d1f9de146e3bc718 not found: ID does not exist" Sep 10 04:43:59.293160 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.293115 2565 scope.go:117] "RemoveContainer" containerID="c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab" Sep 10 04:43:59.293325 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.293303 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab"} err="failed to get container status \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": rpc error: code = NotFound desc = could not find container \"c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab\": container with ID starting with c0eb621337fbae388c390c29387fd8a86daaea353f1dfcefb48a9c89551a46ab not found: ID does not exist" Sep 10 04:43:59.293369 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.293325 2565 scope.go:117] "RemoveContainer" containerID="d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43" Sep 10 04:43:59.293530 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.293513 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43"} err="failed to get container status \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": rpc error: code = NotFound desc = could not find container \"d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43\": container with ID starting with d39d5bee7eac880764accd9a636469d68dd9814aad1bc2416e4464868dbf7d43 not found: ID does not exist" Sep 10 04:43:59.293574 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.293531 2565 scope.go:117] "RemoveContainer" containerID="a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a" Sep 10 04:43:59.293776 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.293760 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a"} err="failed to get container status \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": rpc error: code = NotFound desc = could not find container \"a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a\": container with ID starting with a9e87ae737d31048cc35f6f89ea34a4c662a7a14a25ae17ecd1171b836dacb9a not found: ID does not exist" Sep 10 04:43:59.295923 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.295906 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.298216 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298196 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 04:43:59.298764 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298744 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 04:43:59.298854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298769 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 04:43:59.298854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298770 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bkjk4\"" Sep 10 04:43:59.298854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298818 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 04:43:59.298854 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298826 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 04:43:59.299054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298771 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 04:43:59.299054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298920 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 04:43:59.299054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.298988 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 04:43:59.299628 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.299420 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 04:43:59.299628 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.299471 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 04:43:59.299628 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.299488 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 04:43:59.299628 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.299504 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 04:43:59.299628 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.299539 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dj0g9ulqr16j5\"" Sep 10 04:43:59.301738 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.301720 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 04:43:59.303559 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.303534 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 04:43:59.314207 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.314185 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:43:59.392638 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.392638 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.392858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-web-config\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.392858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392782 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.392858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpjz2\" (UniqueName: \"kubernetes.io/projected/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-kube-api-access-wpjz2\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.392858 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392870 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392913 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-config\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392933 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.392974 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-config-out\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393198 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393027 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393198 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393198 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393198 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393198 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393198 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393124 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393198 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393143 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.393438 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.393199 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.493966 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.493899 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.493966 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.493935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.493966 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.493954 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494183 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.493976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494183 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494001 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494183 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494023 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494183 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494183 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494053 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494183 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494470 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494348 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494470 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-web-config\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494470 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494643 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpjz2\" (UniqueName: \"kubernetes.io/projected/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-kube-api-access-wpjz2\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494643 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494643 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494643 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494585 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494842 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494650 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.494842 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494672 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.495085 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.495061 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.495773 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.495458 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.495773 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.495720 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497226 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497201 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497232 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497241 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.494657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-config\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497233 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-config-out\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497358 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497730 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497730 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497692 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.497841 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.497754 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.498306 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.498278 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-web-config\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.498400 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.498363 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.498582 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.498563 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-config\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.499794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.499774 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.499794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.499785 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.499984 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.499965 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-config-out\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.500723 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.500706 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.503130 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.503110 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpjz2\" (UniqueName: \"kubernetes.io/projected/145c9bfb-a936-4a1c-8d18-1fa3ab91c25e-kube-api-access-wpjz2\") pod \"prometheus-k8s-0\" (UID: \"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.605244 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.605211 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:43:59.883643 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:43:59.883618 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 04:43:59.886022 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:43:59.885993 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod145c9bfb_a936_4a1c_8d18_1fa3ab91c25e.slice/crio-73877b24c52e69116549fb38089421f29779cf4b10362985aa5841368abbb65a WatchSource:0}: Error finding container 73877b24c52e69116549fb38089421f29779cf4b10362985aa5841368abbb65a: Status 404 returned error can't find the container with id 73877b24c52e69116549fb38089421f29779cf4b10362985aa5841368abbb65a Sep 10 04:44:00.235869 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:00.235834 2565 generic.go:356] "Generic (PLEG): container finished" podID="145c9bfb-a936-4a1c-8d18-1fa3ab91c25e" containerID="a056539465790a12fb8baa53de863da26baf66665d04211b980c172aba8631f8" exitCode=0 Sep 10 04:44:00.236055 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:00.235899 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerDied","Data":"a056539465790a12fb8baa53de863da26baf66665d04211b980c172aba8631f8"} Sep 10 04:44:00.236055 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:00.235964 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerStarted","Data":"73877b24c52e69116549fb38089421f29779cf4b10362985aa5841368abbb65a"} Sep 10 04:44:00.391015 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:00.389146 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5846bd9-7fe7-497a-95d8-798518b02028" path="/var/lib/kubelet/pods/d5846bd9-7fe7-497a-95d8-798518b02028/volumes" Sep 10 04:44:01.242849 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:01.242814 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerStarted","Data":"a15bed21147b83b20fcb5ab8261440843846dabe56696dc18d643df6d6c97402"} Sep 10 04:44:01.243220 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:01.242910 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerStarted","Data":"481765d86168722b02665a35a3199eb42a49b498a4fdb7620d5e8a9bbff3ba3f"} Sep 10 04:44:01.243220 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:01.242951 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerStarted","Data":"ae3d9ceddcb01ebd57e8c178da4dee402cc7c652409bc87e84ba4ee0b952ed00"} Sep 10 04:44:01.243220 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:01.242982 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerStarted","Data":"2021e23b22082bc3530f1ce7899edf5562393b0831e06a9a62313f94ef12be4f"} Sep 10 04:44:01.243220 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:01.243005 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerStarted","Data":"1cb308e8e56be81aa62889d11425472d95bcf9f0c78ccf77234be7e6f1d3c818"} Sep 10 04:44:01.243220 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:01.243040 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"145c9bfb-a936-4a1c-8d18-1fa3ab91c25e","Type":"ContainerStarted","Data":"e293d681ed42936b1721ac0e3668221e87bf1705d8405e8fb5cd0ffa5ec4e4d2"} Sep 10 04:44:01.268598 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:01.268547 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.246061026 podStartE2EDuration="2.268533702s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.472676ms" isStatefulPod=false podCreationTimestamp="2026-09-10 04:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:44:01.267121944 +0000 UTC m=+167.362588692" watchObservedRunningTime="2026-09-10 04:44:01.268533702 +0000 UTC m=+167.364000439" Sep 10 04:44:04.606116 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:04.606071 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:44:59.605622 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:59.605554 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:44:59.620544 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:44:59.620522 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:45:00.471730 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:00.471704 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 04:45:53.709142 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.709103 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m68gb"] Sep 10 04:45:53.712492 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.712473 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:53.714892 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.714869 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 04:45:53.714996 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.714953 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 04:45:53.715909 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.715894 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-gg26h\"" Sep 10 04:45:53.719035 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.719006 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m68gb"] Sep 10 04:45:53.870044 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.870009 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4zgc\" (UniqueName: \"kubernetes.io/projected/3af6361c-4bff-4e8b-9939-c68a68616386-kube-api-access-d4zgc\") pod \"cert-manager-cainjector-559cdd8c87-m68gb\" (UID: \"3af6361c-4bff-4e8b-9939-c68a68616386\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:53.870207 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.870057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3af6361c-4bff-4e8b-9939-c68a68616386-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m68gb\" (UID: \"3af6361c-4bff-4e8b-9939-c68a68616386\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:53.971386 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.971298 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4zgc\" (UniqueName: \"kubernetes.io/projected/3af6361c-4bff-4e8b-9939-c68a68616386-kube-api-access-d4zgc\") pod \"cert-manager-cainjector-559cdd8c87-m68gb\" (UID: \"3af6361c-4bff-4e8b-9939-c68a68616386\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:53.971386 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.971350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3af6361c-4bff-4e8b-9939-c68a68616386-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m68gb\" (UID: \"3af6361c-4bff-4e8b-9939-c68a68616386\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:53.980225 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.980200 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3af6361c-4bff-4e8b-9939-c68a68616386-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m68gb\" (UID: \"3af6361c-4bff-4e8b-9939-c68a68616386\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:53.980419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:53.980400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4zgc\" (UniqueName: \"kubernetes.io/projected/3af6361c-4bff-4e8b-9939-c68a68616386-kube-api-access-d4zgc\") pod \"cert-manager-cainjector-559cdd8c87-m68gb\" (UID: \"3af6361c-4bff-4e8b-9939-c68a68616386\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:54.036006 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:54.035970 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" Sep 10 04:45:54.303870 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:54.303831 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m68gb"] Sep 10 04:45:54.306026 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:45:54.305995 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3af6361c_4bff_4e8b_9939_c68a68616386.slice/crio-a1ae5d4966aa5da368896f168994f0ac8d993bff623f77d94f226a0fc240962d WatchSource:0}: Error finding container a1ae5d4966aa5da368896f168994f0ac8d993bff623f77d94f226a0fc240962d: Status 404 returned error can't find the container with id a1ae5d4966aa5da368896f168994f0ac8d993bff623f77d94f226a0fc240962d Sep 10 04:45:54.646386 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:45:54.646302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" event={"ID":"3af6361c-4bff-4e8b-9939-c68a68616386","Type":"ContainerStarted","Data":"a1ae5d4966aa5da368896f168994f0ac8d993bff623f77d94f226a0fc240962d"} Sep 10 04:46:00.671959 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:00.671920 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" event={"ID":"3af6361c-4bff-4e8b-9939-c68a68616386","Type":"ContainerStarted","Data":"21a70bc13b191012a219e703ac41d12cbfd014f342217b145269bc968812da31"} Sep 10 04:46:00.687480 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:00.687425 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m68gb" podStartSLOduration=2.276000831 podStartE2EDuration="7.687408939s" totalImagesPullingTime="5.411408108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:46:00.685459002 +0000 UTC m=+286.780925740" watchObservedRunningTime="2026-09-10 04:46:00.687408939 +0000 UTC m=+286.782875676" Sep 10 04:46:14.263547 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:14.263511 2565 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 04:46:31.377204 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.377169 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945"] Sep 10 04:46:31.426516 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.426492 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945"] Sep 10 04:46:31.426634 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.426582 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.430337 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.430314 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 04:46:31.430337 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.430327 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 04:46:31.430537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.430353 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ghglk\"" Sep 10 04:46:31.430537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.430316 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 04:46:31.430537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.430314 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 04:46:31.430537 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.430374 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 04:46:31.560399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.560361 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7ssj\" (UniqueName: \"kubernetes.io/projected/1361a7cf-1311-452f-92e7-b015d62ca293-kube-api-access-w7ssj\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.560399 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.560404 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1361a7cf-1311-452f-92e7-b015d62ca293-manager-config\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.560595 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.560423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1361a7cf-1311-452f-92e7-b015d62ca293-metrics-cert\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.560595 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.560520 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1361a7cf-1311-452f-92e7-b015d62ca293-cert\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.661326 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.661243 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7ssj\" (UniqueName: \"kubernetes.io/projected/1361a7cf-1311-452f-92e7-b015d62ca293-kube-api-access-w7ssj\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.661726 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.661695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1361a7cf-1311-452f-92e7-b015d62ca293-manager-config\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.661865 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.661742 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1361a7cf-1311-452f-92e7-b015d62ca293-metrics-cert\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.661865 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.661792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1361a7cf-1311-452f-92e7-b015d62ca293-cert\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.662427 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.662401 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1361a7cf-1311-452f-92e7-b015d62ca293-manager-config\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.664351 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.664330 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1361a7cf-1311-452f-92e7-b015d62ca293-cert\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.664448 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.664386 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1361a7cf-1311-452f-92e7-b015d62ca293-metrics-cert\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.676995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.676965 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7ssj\" (UniqueName: \"kubernetes.io/projected/1361a7cf-1311-452f-92e7-b015d62ca293-kube-api-access-w7ssj\") pod \"lws-controller-manager-64cfcb4bb4-sr945\" (UID: \"1361a7cf-1311-452f-92e7-b015d62ca293\") " pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:31.735421 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:31.735385 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:32.008907 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:32.008875 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945"] Sep 10 04:46:32.012474 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:46:32.012450 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1361a7cf_1311_452f_92e7_b015d62ca293.slice/crio-1dc64e088749e39da2349dd8b52d391e354ab04a3519d97396eb835e46fff4cc WatchSource:0}: Error finding container 1dc64e088749e39da2349dd8b52d391e354ab04a3519d97396eb835e46fff4cc: Status 404 returned error can't find the container with id 1dc64e088749e39da2349dd8b52d391e354ab04a3519d97396eb835e46fff4cc Sep 10 04:46:32.015301 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:32.015279 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 04:46:32.795579 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:32.795510 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" event={"ID":"1361a7cf-1311-452f-92e7-b015d62ca293","Type":"ContainerStarted","Data":"1dc64e088749e39da2349dd8b52d391e354ab04a3519d97396eb835e46fff4cc"} Sep 10 04:46:34.804641 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:34.804594 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" event={"ID":"1361a7cf-1311-452f-92e7-b015d62ca293","Type":"ContainerStarted","Data":"298f83d9e500594efc412f490c486ca15b5031e248bf4ec32cdfc2661c5e3bb8"} Sep 10 04:46:34.805020 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:34.804722 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:34.821714 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:34.821666 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" podStartSLOduration=1.494306815 podStartE2EDuration="3.821650253s" totalImagesPullingTime="2.327343438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:46:34.820186266 +0000 UTC m=+320.915653026" watchObservedRunningTime="2026-09-10 04:46:34.821650253 +0000 UTC m=+320.917116987" Sep 10 04:46:35.871420 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.871382 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh"] Sep 10 04:46:35.874636 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.874594 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:35.877523 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.877500 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 04:46:35.877523 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.877516 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 04:46:35.877861 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.877844 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2nmgk\"" Sep 10 04:46:35.877945 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.877848 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 04:46:35.881275 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.881253 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 04:46:35.888422 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:35.888400 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh"] Sep 10 04:46:36.001740 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.001702 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvn4l\" (UniqueName: \"kubernetes.io/projected/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-kube-api-access-bvn4l\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.001980 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.001757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-manifests\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.001980 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.001779 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-tmp\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.001980 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.001820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.001980 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.001862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.102267 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.102232 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvn4l\" (UniqueName: \"kubernetes.io/projected/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-kube-api-access-bvn4l\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.102448 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.102285 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-manifests\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.102448 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.102306 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-tmp\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.102448 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.102325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.102448 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.102340 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.102709 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.102671 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-manifests\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.104531 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.104511 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-tmp\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.104812 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.104790 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.104875 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.104849 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.110927 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.110900 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvn4l\" (UniqueName: \"kubernetes.io/projected/6e0adb95-b2fe-48f2-8dcb-9699a8214fa0-kube-api-access-bvn4l\") pod \"opendatahub-operator-controller-manager-6766b6589c-dvvkh\" (UID: \"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0\") " pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.184933 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.184898 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:36.475721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.475698 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh"] Sep 10 04:46:36.478214 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:46:36.478188 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0adb95_b2fe_48f2_8dcb_9699a8214fa0.slice/crio-0c316f2b6e720f2d8f623803dd6940093df7bf91d2f77341185f4adf5ef180b0 WatchSource:0}: Error finding container 0c316f2b6e720f2d8f623803dd6940093df7bf91d2f77341185f4adf5ef180b0: Status 404 returned error can't find the container with id 0c316f2b6e720f2d8f623803dd6940093df7bf91d2f77341185f4adf5ef180b0 Sep 10 04:46:36.813794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:36.813712 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" event={"ID":"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0","Type":"ContainerStarted","Data":"0c316f2b6e720f2d8f623803dd6940093df7bf91d2f77341185f4adf5ef180b0"} Sep 10 04:46:39.829096 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:39.829016 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" event={"ID":"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0","Type":"ContainerStarted","Data":"d54f6c3f5e9178576d2d24a6634adef7ed86c19e0c2b0f097c8014b84e045346"} Sep 10 04:46:40.834984 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:40.834951 2565 generic.go:356] "Generic (PLEG): container finished" podID="6e0adb95-b2fe-48f2-8dcb-9699a8214fa0" containerID="d54f6c3f5e9178576d2d24a6634adef7ed86c19e0c2b0f097c8014b84e045346" exitCode=0 Sep 10 04:46:40.835329 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:40.835039 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" event={"ID":"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0","Type":"ContainerDied","Data":"d54f6c3f5e9178576d2d24a6634adef7ed86c19e0c2b0f097c8014b84e045346"} Sep 10 04:46:41.841086 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:41.841046 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" event={"ID":"6e0adb95-b2fe-48f2-8dcb-9699a8214fa0","Type":"ContainerStarted","Data":"00b8749d57804bdd41f5271384356638fc3676cb2d486780514816427113c49b"} Sep 10 04:46:41.841516 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:41.841165 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:41.862357 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:41.862296 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" podStartSLOduration=2.308582343 podStartE2EDuration="6.86227818s" totalImagesPullingTime="3.47694075s" totalInitContainerRuntime="1.076755087s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:46:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:46:41.860973087 +0000 UTC m=+327.956439849" watchObservedRunningTime="2026-09-10 04:46:41.86227818 +0000 UTC m=+327.957744917" Sep 10 04:46:45.810269 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:45.810239 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64cfcb4bb4-sr945" Sep 10 04:46:52.847367 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:52.847339 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6766b6589c-dvvkh" Sep 10 04:46:58.402849 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.402814 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z"] Sep 10 04:46:58.411581 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.411559 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.414321 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.414296 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 04:46:58.415282 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.414541 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 04:46:58.415531 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.414664 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 04:46:58.415650 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.414974 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 04:46:58.415650 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.415014 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hgwr8\"" Sep 10 04:46:58.416249 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.416231 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z"] Sep 10 04:46:58.478765 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.478732 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skw4w\" (UniqueName: \"kubernetes.io/projected/46b7b3fd-811e-4ca3-be0d-346acd7a5490-kube-api-access-skw4w\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.478887 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.478781 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tls-certs\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.478887 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.478858 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tmp\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.579505 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.579476 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tls-certs\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.579671 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.579533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tmp\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.579671 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.579582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skw4w\" (UniqueName: \"kubernetes.io/projected/46b7b3fd-811e-4ca3-be0d-346acd7a5490-kube-api-access-skw4w\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.579671 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:46:58.579657 2565 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 10 04:46:58.579797 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:46:58.579734 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tls-certs podName:46b7b3fd-811e-4ca3-be0d-346acd7a5490 nodeName:}" failed. No retries permitted until 2026-09-10 04:46:59.079711053 +0000 UTC m=+345.175177785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tls-certs") pod "kube-auth-proxy-7cbb8ff44d-m9l5z" (UID: "46b7b3fd-811e-4ca3-be0d-346acd7a5490") : secret "kube-auth-proxy-tls" not found Sep 10 04:46:58.581760 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.581738 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tmp\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:58.588973 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:58.588954 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skw4w\" (UniqueName: \"kubernetes.io/projected/46b7b3fd-811e-4ca3-be0d-346acd7a5490-kube-api-access-skw4w\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:59.083884 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:59.083852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tls-certs\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:59.086289 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:59.086260 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b7b3fd-811e-4ca3-be0d-346acd7a5490-tls-certs\") pod \"kube-auth-proxy-7cbb8ff44d-m9l5z\" (UID: \"46b7b3fd-811e-4ca3-be0d-346acd7a5490\") " pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:59.323794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:59.323756 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" Sep 10 04:46:59.597983 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:59.597959 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z"] Sep 10 04:46:59.600416 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:46:59.600382 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b7b3fd_811e_4ca3_be0d_346acd7a5490.slice/crio-9733f6ce2b309c8b64de40e115e9e52f3c7a33a811cae2dbedf5af3a7ff76a69 WatchSource:0}: Error finding container 9733f6ce2b309c8b64de40e115e9e52f3c7a33a811cae2dbedf5af3a7ff76a69: Status 404 returned error can't find the container with id 9733f6ce2b309c8b64de40e115e9e52f3c7a33a811cae2dbedf5af3a7ff76a69 Sep 10 04:46:59.921011 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:46:59.920976 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" event={"ID":"46b7b3fd-811e-4ca3-be0d-346acd7a5490","Type":"ContainerStarted","Data":"9733f6ce2b309c8b64de40e115e9e52f3c7a33a811cae2dbedf5af3a7ff76a69"} Sep 10 04:47:02.936320 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:47:02.936285 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" event={"ID":"46b7b3fd-811e-4ca3-be0d-346acd7a5490","Type":"ContainerStarted","Data":"30c70f0b379d73f7c9cd0069c043760879ceadf8f2bb2e45e0db8a3f97834691"} Sep 10 04:47:02.952768 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:47:02.952717 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cbb8ff44d-m9l5z" podStartSLOduration=2.155052909 podStartE2EDuration="4.952700554s" totalImagesPullingTime="2.797647645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:47:02.952400912 +0000 UTC m=+349.047867699" watchObservedRunningTime="2026-09-10 04:47:02.952700554 +0000 UTC m=+349.048167291" Sep 10 04:48:34.673853 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.673813 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7"] Sep 10 04:48:34.677133 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.677114 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" Sep 10 04:48:34.679777 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.679752 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 10 04:48:34.679897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.679752 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 04:48:34.679897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.679794 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qmlbv\"" Sep 10 04:48:34.679897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.679848 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 04:48:34.692910 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.692886 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7"] Sep 10 04:48:34.753897 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.753860 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d66bg\" (UniqueName: \"kubernetes.io/projected/3ab35c66-2f40-47a6-8a81-67b3a81e17ba-kube-api-access-d66bg\") pod \"dns-operator-controller-manager-7b84cd7c9c-jcpk7\" (UID: \"3ab35c66-2f40-47a6-8a81-67b3a81e17ba\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" Sep 10 04:48:34.855224 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.855190 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d66bg\" (UniqueName: \"kubernetes.io/projected/3ab35c66-2f40-47a6-8a81-67b3a81e17ba-kube-api-access-d66bg\") pod \"dns-operator-controller-manager-7b84cd7c9c-jcpk7\" (UID: \"3ab35c66-2f40-47a6-8a81-67b3a81e17ba\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" Sep 10 04:48:34.864213 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.864189 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d66bg\" (UniqueName: \"kubernetes.io/projected/3ab35c66-2f40-47a6-8a81-67b3a81e17ba-kube-api-access-d66bg\") pod \"dns-operator-controller-manager-7b84cd7c9c-jcpk7\" (UID: \"3ab35c66-2f40-47a6-8a81-67b3a81e17ba\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" Sep 10 04:48:34.987350 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:34.987254 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" Sep 10 04:48:35.259536 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:35.259513 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7"] Sep 10 04:48:35.262054 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:48:35.262031 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab35c66_2f40_47a6_8a81_67b3a81e17ba.slice/crio-ee8b103010da159fcf95a8223793a5055d2b4eeec0e41485dc36b32445056788 WatchSource:0}: Error finding container ee8b103010da159fcf95a8223793a5055d2b4eeec0e41485dc36b32445056788: Status 404 returned error can't find the container with id ee8b103010da159fcf95a8223793a5055d2b4eeec0e41485dc36b32445056788 Sep 10 04:48:35.312337 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:35.312303 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" event={"ID":"3ab35c66-2f40-47a6-8a81-67b3a81e17ba","Type":"ContainerStarted","Data":"ee8b103010da159fcf95a8223793a5055d2b4eeec0e41485dc36b32445056788"} Sep 10 04:48:38.328529 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:38.328490 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" event={"ID":"3ab35c66-2f40-47a6-8a81-67b3a81e17ba","Type":"ContainerStarted","Data":"503e947487aef816e94aa135776a12e78bfb6ba18633d2786505790b09bedc89"} Sep 10 04:48:38.329020 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:38.328634 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" Sep 10 04:48:38.345659 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:38.345543 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" podStartSLOduration=1.522963919 podStartE2EDuration="4.345526842s" totalImagesPullingTime="2.822562923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:48:38.344808585 +0000 UTC m=+444.440275344" watchObservedRunningTime="2026-09-10 04:48:38.345526842 +0000 UTC m=+444.440993599" Sep 10 04:48:44.962762 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:44.962727 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m"] Sep 10 04:48:44.966065 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:44.966035 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:44.970085 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:44.970063 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gzrl5\"" Sep 10 04:48:44.980408 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:44.980377 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m"] Sep 10 04:48:45.142380 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.142339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b935a742-14b2-4966-a66b-241f6ad8bd23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.142552 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.142391 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfjzq\" (UniqueName: \"kubernetes.io/projected/b935a742-14b2-4966-a66b-241f6ad8bd23-kube-api-access-gfjzq\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.242993 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.242923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b935a742-14b2-4966-a66b-241f6ad8bd23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.242993 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.242973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfjzq\" (UniqueName: \"kubernetes.io/projected/b935a742-14b2-4966-a66b-241f6ad8bd23-kube-api-access-gfjzq\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.243395 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.243374 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b935a742-14b2-4966-a66b-241f6ad8bd23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.250760 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.250735 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfjzq\" (UniqueName: \"kubernetes.io/projected/b935a742-14b2-4966-a66b-241f6ad8bd23-kube-api-access-gfjzq\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.275758 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.275728 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.457515 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.457478 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m"] Sep 10 04:48:45.461623 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.461568 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m"] Sep 10 04:48:45.490848 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.490815 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc"] Sep 10 04:48:45.494418 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.494358 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:45.519422 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.519393 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc"] Sep 10 04:48:45.646166 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.646128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngthq\" (UniqueName: \"kubernetes.io/projected/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-kube-api-access-ngthq\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-l6wrc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:45.646332 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.646199 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-l6wrc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:45.661200 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:48:45.661166 2565 log.go:32] "RunPodSandbox from runtime service failed" err=< Sep 10 04:48:45.661200 ip-10-0-136-230 kubenswrapper[2565]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-7mp9m_kuadrant-system_b935a742-14b2-4966-a66b-241f6ad8bd23_0(d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-7mp9m to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293" Netns:"/var/run/netns/d3226de1-1f20-46cb-ac16-9ae803edfa8e" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-7mp9m;K8S_POD_INFRA_CONTAINER_ID=d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293;K8S_POD_UID=b935a742-14b2-4966-a66b-241f6ad8bd23" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m/b935a742-14b2-4966-a66b-241f6ad8bd23:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:19 [10.133.0.25/23] Sep 10 04:48:45.661200 ip-10-0-136-230 kubenswrapper[2565]: ' Sep 10 04:48:45.661200 ip-10-0-136-230 kubenswrapper[2565]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Sep 10 04:48:45.661200 ip-10-0-136-230 kubenswrapper[2565]: > Sep 10 04:48:45.661431 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:48:45.661225 2565 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Sep 10 04:48:45.661431 ip-10-0-136-230 kubenswrapper[2565]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-7mp9m_kuadrant-system_b935a742-14b2-4966-a66b-241f6ad8bd23_0(d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-7mp9m to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293" Netns:"/var/run/netns/d3226de1-1f20-46cb-ac16-9ae803edfa8e" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-7mp9m;K8S_POD_INFRA_CONTAINER_ID=d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293;K8S_POD_UID=b935a742-14b2-4966-a66b-241f6ad8bd23" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m/b935a742-14b2-4966-a66b-241f6ad8bd23:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m d580620e51656dac9a0c3642e778427bd53057bcc7332b5fc1d38996dba81293 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:19 [10.133.0.25/23] Sep 10 04:48:45.661431 ip-10-0-136-230 kubenswrapper[2565]: ' Sep 10 04:48:45.661431 ip-10-0-136-230 kubenswrapper[2565]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Sep 10 04:48:45.661431 ip-10-0-136-230 kubenswrapper[2565]: > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:45.747715 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.747624 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngthq\" (UniqueName: \"kubernetes.io/projected/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-kube-api-access-ngthq\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-l6wrc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:45.747715 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.747682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-l6wrc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:45.748026 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.748010 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-l6wrc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:45.759916 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.759889 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngthq\" (UniqueName: \"kubernetes.io/projected/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-kube-api-access-ngthq\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-l6wrc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:45.805500 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:45.805456 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:46.078729 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.078701 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc"] Sep 10 04:48:46.081023 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:48:46.080994 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeafd2bb_e278_41d8_ac3b_ecf42c1e55fc.slice/crio-b6e7e0e3bc86e8c8561cc119fa854762ca4d6e55b80398ffc186a7c868a11fe8 WatchSource:0}: Error finding container b6e7e0e3bc86e8c8561cc119fa854762ca4d6e55b80398ffc186a7c868a11fe8: Status 404 returned error can't find the container with id b6e7e0e3bc86e8c8561cc119fa854762ca4d6e55b80398ffc186a7c868a11fe8 Sep 10 04:48:46.363127 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.363043 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" event={"ID":"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc","Type":"ContainerStarted","Data":"b6e7e0e3bc86e8c8561cc119fa854762ca4d6e55b80398ffc186a7c868a11fe8"} Sep 10 04:48:46.363127 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.363059 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:46.365258 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:48:46.365233 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" is forbidden: User \"system:node:ip-10-0-136-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-230.ec2.internal' and this object" podUID="b935a742-14b2-4966-a66b-241f6ad8bd23" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:46.370778 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.370759 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:46.372546 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:48:46.372526 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" is forbidden: User \"system:node:ip-10-0-136-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-230.ec2.internal' and this object" podUID="b935a742-14b2-4966-a66b-241f6ad8bd23" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:46.556656 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.556627 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b935a742-14b2-4966-a66b-241f6ad8bd23-extensions-socket-volume\") pod \"b935a742-14b2-4966-a66b-241f6ad8bd23\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " Sep 10 04:48:46.556873 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.556698 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfjzq\" (UniqueName: \"kubernetes.io/projected/b935a742-14b2-4966-a66b-241f6ad8bd23-kube-api-access-gfjzq\") pod \"b935a742-14b2-4966-a66b-241f6ad8bd23\" (UID: \"b935a742-14b2-4966-a66b-241f6ad8bd23\") " Sep 10 04:48:46.556946 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.556920 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b935a742-14b2-4966-a66b-241f6ad8bd23-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b935a742-14b2-4966-a66b-241f6ad8bd23" (UID: "b935a742-14b2-4966-a66b-241f6ad8bd23"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 04:48:46.558751 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.558721 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b935a742-14b2-4966-a66b-241f6ad8bd23-kube-api-access-gfjzq" (OuterVolumeSpecName: "kube-api-access-gfjzq") pod "b935a742-14b2-4966-a66b-241f6ad8bd23" (UID: "b935a742-14b2-4966-a66b-241f6ad8bd23"). InnerVolumeSpecName "kube-api-access-gfjzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:48:46.657543 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.657466 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfjzq\" (UniqueName: \"kubernetes.io/projected/b935a742-14b2-4966-a66b-241f6ad8bd23-kube-api-access-gfjzq\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:48:46.657543 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:46.657495 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b935a742-14b2-4966-a66b-241f6ad8bd23-extensions-socket-volume\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:48:47.367107 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:47.367075 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:47.374649 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:48:47.374616 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-7mp9m\" is forbidden: User \"system:node:ip-10-0-136-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-230.ec2.internal' and this object" podUID="b935a742-14b2-4966-a66b-241f6ad8bd23" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-7mp9m" Sep 10 04:48:48.385390 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:48.385352 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b935a742-14b2-4966-a66b-241f6ad8bd23" path="/var/lib/kubelet/pods/b935a742-14b2-4966-a66b-241f6ad8bd23/volumes" Sep 10 04:48:49.335619 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:49.335570 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-jcpk7" Sep 10 04:48:51.386940 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:51.386901 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" event={"ID":"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc","Type":"ContainerStarted","Data":"d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb"} Sep 10 04:48:51.387368 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:51.386982 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:48:51.407328 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:48:51.407282 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" podStartSLOduration=1.749034824 podStartE2EDuration="6.407267419s" totalImagesPullingTime="4.658232595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:48:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:48:51.406404812 +0000 UTC m=+457.501871548" watchObservedRunningTime="2026-09-10 04:48:51.407267419 +0000 UTC m=+457.502734157" Sep 10 04:49:02.392637 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:02.392590 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:49:14.554707 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:14.554676 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc"] Sep 10 04:49:14.555090 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:14.554908 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" podUID="aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" containerName="manager" containerID="cri-o://d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb" gracePeriod=10 Sep 10 04:49:14.797435 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:14.797413 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:49:14.899086 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:14.899015 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-extensions-socket-volume\") pod \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " Sep 10 04:49:14.899086 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:14.899064 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngthq\" (UniqueName: \"kubernetes.io/projected/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-kube-api-access-ngthq\") pod \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\" (UID: \"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc\") " Sep 10 04:49:14.899370 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:14.899346 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" (UID: "aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 04:49:14.901203 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:14.901184 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-kube-api-access-ngthq" (OuterVolumeSpecName: "kube-api-access-ngthq") pod "aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" (UID: "aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc"). InnerVolumeSpecName "kube-api-access-ngthq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:49:15.000215 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.000179 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngthq\" (UniqueName: \"kubernetes.io/projected/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-kube-api-access-ngthq\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:49:15.000215 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.000207 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc-extensions-socket-volume\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:49:15.489391 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.489362 2565 generic.go:356] "Generic (PLEG): container finished" podID="aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" containerID="d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb" exitCode=0 Sep 10 04:49:15.489556 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.489406 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" Sep 10 04:49:15.489556 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.489416 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" event={"ID":"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc","Type":"ContainerDied","Data":"d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb"} Sep 10 04:49:15.489556 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.489459 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc" event={"ID":"aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc","Type":"ContainerDied","Data":"b6e7e0e3bc86e8c8561cc119fa854762ca4d6e55b80398ffc186a7c868a11fe8"} Sep 10 04:49:15.489556 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.489480 2565 scope.go:117] "RemoveContainer" containerID="d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb" Sep 10 04:49:15.497533 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.497512 2565 scope.go:117] "RemoveContainer" containerID="d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb" Sep 10 04:49:15.497830 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:49:15.497809 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb\": container with ID starting with d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb not found: ID does not exist" containerID="d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb" Sep 10 04:49:15.497902 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.497838 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb"} err="failed to get container status \"d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb\": rpc error: code = NotFound desc = could not find container \"d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb\": container with ID starting with d85013f9322610bd32dde423922637957cd39a690814e04725d4a3acdb1551eb not found: ID does not exist" Sep 10 04:49:15.513813 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.513793 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc"] Sep 10 04:49:15.520286 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:15.520267 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-l6wrc"] Sep 10 04:49:16.385083 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:16.385052 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" path="/var/lib/kubelet/pods/aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc/volumes" Sep 10 04:49:24.052465 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.052431 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:49:24.052883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.052773 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" containerName="manager" Sep 10 04:49:24.052883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.052784 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" containerName="manager" Sep 10 04:49:24.052883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.052838 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeafd2bb-e278-41d8-ac3b-ecf42c1e55fc" containerName="manager" Sep 10 04:49:24.058265 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.058243 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.061530 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.061506 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pjzgs\"" Sep 10 04:49:24.061653 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.061556 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 10 04:49:24.071558 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.071535 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:49:24.118264 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.118229 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:49:24.186041 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.186004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccr45\" (UniqueName: \"kubernetes.io/projected/7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f-kube-api-access-ccr45\") pod \"limitador-limitador-84994f7ffc-cv7r9\" (UID: \"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.186041 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.186045 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f-config-file\") pod \"limitador-limitador-84994f7ffc-cv7r9\" (UID: \"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.286915 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.286876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccr45\" (UniqueName: \"kubernetes.io/projected/7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f-kube-api-access-ccr45\") pod \"limitador-limitador-84994f7ffc-cv7r9\" (UID: \"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.286915 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.286918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f-config-file\") pod \"limitador-limitador-84994f7ffc-cv7r9\" (UID: \"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.287508 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.287488 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f-config-file\") pod \"limitador-limitador-84994f7ffc-cv7r9\" (UID: \"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.296126 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.296106 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccr45\" (UniqueName: \"kubernetes.io/projected/7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f-kube-api-access-ccr45\") pod \"limitador-limitador-84994f7ffc-cv7r9\" (UID: \"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.368210 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.368122 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:24.542615 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.542581 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-n45v8"] Sep 10 04:49:24.546533 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.546518 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-n45v8" Sep 10 04:49:24.549632 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.549143 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ffrwm\"" Sep 10 04:49:24.556983 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.556954 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-n45v8"] Sep 10 04:49:24.640539 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.640512 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:49:24.643067 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:49:24.643041 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b8c5fd5_3ef7_4034_ba9c_9a73ddf2905f.slice/crio-8e78efbec730946d40a3c59db960edd7dbcc16a5fc6ad378536e22162d36f226 WatchSource:0}: Error finding container 8e78efbec730946d40a3c59db960edd7dbcc16a5fc6ad378536e22162d36f226: Status 404 returned error can't find the container with id 8e78efbec730946d40a3c59db960edd7dbcc16a5fc6ad378536e22162d36f226 Sep 10 04:49:24.690244 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.690209 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5sb2\" (UniqueName: \"kubernetes.io/projected/7b2ae00b-318a-459c-a647-2c194f11c547-kube-api-access-v5sb2\") pod \"authorino-59c974bd48-n45v8\" (UID: \"7b2ae00b-318a-459c-a647-2c194f11c547\") " pod="kuadrant-system/authorino-59c974bd48-n45v8" Sep 10 04:49:24.790995 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.790960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5sb2\" (UniqueName: \"kubernetes.io/projected/7b2ae00b-318a-459c-a647-2c194f11c547-kube-api-access-v5sb2\") pod \"authorino-59c974bd48-n45v8\" (UID: \"7b2ae00b-318a-459c-a647-2c194f11c547\") " pod="kuadrant-system/authorino-59c974bd48-n45v8" Sep 10 04:49:24.803277 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.803241 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5sb2\" (UniqueName: \"kubernetes.io/projected/7b2ae00b-318a-459c-a647-2c194f11c547-kube-api-access-v5sb2\") pod \"authorino-59c974bd48-n45v8\" (UID: \"7b2ae00b-318a-459c-a647-2c194f11c547\") " pod="kuadrant-system/authorino-59c974bd48-n45v8" Sep 10 04:49:24.861187 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:24.861151 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-n45v8" Sep 10 04:49:25.131003 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:25.130914 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-n45v8"] Sep 10 04:49:25.133552 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:49:25.133526 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b2ae00b_318a_459c_a647_2c194f11c547.slice/crio-75563e7fdd16a5913a015bb8eb41f38c5528789bd60bf2eea193bed74c1d2d7a WatchSource:0}: Error finding container 75563e7fdd16a5913a015bb8eb41f38c5528789bd60bf2eea193bed74c1d2d7a: Status 404 returned error can't find the container with id 75563e7fdd16a5913a015bb8eb41f38c5528789bd60bf2eea193bed74c1d2d7a Sep 10 04:49:25.537009 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:25.536975 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-n45v8" event={"ID":"7b2ae00b-318a-459c-a647-2c194f11c547","Type":"ContainerStarted","Data":"75563e7fdd16a5913a015bb8eb41f38c5528789bd60bf2eea193bed74c1d2d7a"} Sep 10 04:49:25.538319 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:25.538281 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" event={"ID":"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f","Type":"ContainerStarted","Data":"8e78efbec730946d40a3c59db960edd7dbcc16a5fc6ad378536e22162d36f226"} Sep 10 04:49:26.544578 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:26.544542 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" event={"ID":"7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f","Type":"ContainerStarted","Data":"41468afbd26cc61ba4cb5f4b25f8f77429250cf4702147aef3eb5c96f0ae1538"} Sep 10 04:49:26.545062 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:26.544676 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:26.562664 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:26.562597 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" podStartSLOduration=1.371321821 podStartE2EDuration="2.562581682s" totalImagesPullingTime="1.191259861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:49:26.561123832 +0000 UTC m=+492.656590590" watchObservedRunningTime="2026-09-10 04:49:26.562581682 +0000 UTC m=+492.658048419" Sep 10 04:49:27.551944 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:27.551900 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-n45v8" event={"ID":"7b2ae00b-318a-459c-a647-2c194f11c547","Type":"ContainerStarted","Data":"f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee"} Sep 10 04:49:27.566683 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:27.566636 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-n45v8" podStartSLOduration=1.869682464 podStartE2EDuration="3.56662016s" totalImagesPullingTime="1.696937696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:49:27.56552156 +0000 UTC m=+493.660988300" watchObservedRunningTime="2026-09-10 04:49:27.56662016 +0000 UTC m=+493.662086888" Sep 10 04:49:37.552882 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:37.552839 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-cv7r9" Sep 10 04:49:52.594330 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.594295 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59b7d9c9fd-cwwln"] Sep 10 04:49:52.597567 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.597547 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" Sep 10 04:49:52.606728 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.606701 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59b7d9c9fd-cwwln"] Sep 10 04:49:52.619510 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.619481 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp2df\" (UniqueName: \"kubernetes.io/projected/8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc-kube-api-access-rp2df\") pod \"authorino-59b7d9c9fd-cwwln\" (UID: \"8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc\") " pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" Sep 10 04:49:52.637466 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.637438 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59b7d9c9fd-cwwln"] Sep 10 04:49:52.637707 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:49:52.637687 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rp2df], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" podUID="8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc" Sep 10 04:49:52.658626 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.658584 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" Sep 10 04:49:52.665098 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.665077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" Sep 10 04:49:52.720938 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.720906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp2df\" (UniqueName: \"kubernetes.io/projected/8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc-kube-api-access-rp2df\") pod \"authorino-59b7d9c9fd-cwwln\" (UID: \"8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc\") " pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" Sep 10 04:49:52.728929 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.728902 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp2df\" (UniqueName: \"kubernetes.io/projected/8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc-kube-api-access-rp2df\") pod \"authorino-59b7d9c9fd-cwwln\" (UID: \"8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc\") " pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" Sep 10 04:49:52.821927 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.821892 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp2df\" (UniqueName: \"kubernetes.io/projected/8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc-kube-api-access-rp2df\") pod \"8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc\" (UID: \"8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc\") " Sep 10 04:49:52.823961 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.823935 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc-kube-api-access-rp2df" (OuterVolumeSpecName: "kube-api-access-rp2df") pod "8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc" (UID: "8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc"). InnerVolumeSpecName "kube-api-access-rp2df". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:49:52.923267 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:52.923228 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp2df\" (UniqueName: \"kubernetes.io/projected/8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc-kube-api-access-rp2df\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:49:53.662939 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:53.662909 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b7d9c9fd-cwwln" Sep 10 04:49:53.692303 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:53.692275 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59b7d9c9fd-cwwln"] Sep 10 04:49:53.694811 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:53.694787 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59b7d9c9fd-cwwln"] Sep 10 04:49:54.384708 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:54.384677 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc" path="/var/lib/kubelet/pods/8b8bf9b2-d6fb-44b5-bc52-55e127a1fdcc/volumes" Sep 10 04:49:54.876490 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:54.876458 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-n45v8"] Sep 10 04:49:54.876885 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:54.876648 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-n45v8" podUID="7b2ae00b-318a-459c-a647-2c194f11c547" containerName="authorino" containerID="cri-o://f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee" gracePeriod=30 Sep 10 04:49:55.106376 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.106350 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-n45v8" Sep 10 04:49:55.142245 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.142183 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5sb2\" (UniqueName: \"kubernetes.io/projected/7b2ae00b-318a-459c-a647-2c194f11c547-kube-api-access-v5sb2\") pod \"7b2ae00b-318a-459c-a647-2c194f11c547\" (UID: \"7b2ae00b-318a-459c-a647-2c194f11c547\") " Sep 10 04:49:55.144143 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.144118 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b2ae00b-318a-459c-a647-2c194f11c547-kube-api-access-v5sb2" (OuterVolumeSpecName: "kube-api-access-v5sb2") pod "7b2ae00b-318a-459c-a647-2c194f11c547" (UID: "7b2ae00b-318a-459c-a647-2c194f11c547"). InnerVolumeSpecName "kube-api-access-v5sb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:49:55.243080 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.243043 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5sb2\" (UniqueName: \"kubernetes.io/projected/7b2ae00b-318a-459c-a647-2c194f11c547-kube-api-access-v5sb2\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:49:55.672386 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.672347 2565 generic.go:356] "Generic (PLEG): container finished" podID="7b2ae00b-318a-459c-a647-2c194f11c547" containerID="f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee" exitCode=0 Sep 10 04:49:55.672560 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.672392 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-n45v8" Sep 10 04:49:55.672560 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.672433 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-n45v8" event={"ID":"7b2ae00b-318a-459c-a647-2c194f11c547","Type":"ContainerDied","Data":"f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee"} Sep 10 04:49:55.672560 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.672472 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-n45v8" event={"ID":"7b2ae00b-318a-459c-a647-2c194f11c547","Type":"ContainerDied","Data":"75563e7fdd16a5913a015bb8eb41f38c5528789bd60bf2eea193bed74c1d2d7a"} Sep 10 04:49:55.672560 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.672489 2565 scope.go:117] "RemoveContainer" containerID="f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee" Sep 10 04:49:55.680567 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.680550 2565 scope.go:117] "RemoveContainer" containerID="f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee" Sep 10 04:49:55.680879 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:49:55.680861 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee\": container with ID starting with f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee not found: ID does not exist" containerID="f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee" Sep 10 04:49:55.680938 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.680886 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee"} err="failed to get container status \"f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee\": rpc error: code = NotFound desc = could not find container \"f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee\": container with ID starting with f8ef01bbe0bb2b9daa764b63ef3a61f044241af1a949bc7d8ab65311c7a78bee not found: ID does not exist" Sep 10 04:49:55.691917 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.691899 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-n45v8"] Sep 10 04:49:55.695581 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:55.695559 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-n45v8"] Sep 10 04:49:56.384462 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:56.384428 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b2ae00b-318a-459c-a647-2c194f11c547" path="/var/lib/kubelet/pods/7b2ae00b-318a-459c-a647-2c194f11c547/volumes" Sep 10 04:49:59.857438 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.857400 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5cd6fb4d4f-sn2fc"] Sep 10 04:49:59.858004 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.857882 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b2ae00b-318a-459c-a647-2c194f11c547" containerName="authorino" Sep 10 04:49:59.858004 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.857901 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b2ae00b-318a-459c-a647-2c194f11c547" containerName="authorino" Sep 10 04:49:59.858122 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.858005 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b2ae00b-318a-459c-a647-2c194f11c547" containerName="authorino" Sep 10 04:49:59.863465 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.863445 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:49:59.868041 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.868020 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 04:49:59.868241 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.868227 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 04:49:59.868949 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.868929 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m8ql9\"" Sep 10 04:49:59.869034 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.868957 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 04:49:59.882430 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.882403 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cd6fb4d4f-sn2fc"] Sep 10 04:49:59.882783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.882759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:49:59.882851 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.882835 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:49:59.882906 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.882893 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frcxn\" (UniqueName: \"kubernetes.io/projected/f5938821-854c-426f-8023-66b141804fe7-kube-api-access-frcxn\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:49:59.984075 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.984031 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frcxn\" (UniqueName: \"kubernetes.io/projected/f5938821-854c-426f-8023-66b141804fe7-kube-api-access-frcxn\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:49:59.984275 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.984089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:49:59.984275 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.984129 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:49:59.984275 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:49:59.984235 2565 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 10 04:49:59.984275 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:49:59.984266 2565 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 10 04:49:59.984482 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:49:59.984292 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert podName:f5938821-854c-426f-8023-66b141804fe7 nodeName:}" failed. No retries permitted until 2026-09-10 04:50:00.484273474 +0000 UTC m=+526.579740191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert") pod "maas-controller-5cd6fb4d4f-sn2fc" (UID: "f5938821-854c-426f-8023-66b141804fe7") : secret "maas-controller-webhook-cert" not found Sep 10 04:49:59.984482 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:49:59.984319 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs podName:f5938821-854c-426f-8023-66b141804fe7 nodeName:}" failed. No retries permitted until 2026-09-10 04:50:00.484302726 +0000 UTC m=+526.579769446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs") pod "maas-controller-5cd6fb4d4f-sn2fc" (UID: "f5938821-854c-426f-8023-66b141804fe7") : secret "maas-controller-metrics-tls" not found Sep 10 04:49:59.992959 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:49:59.992928 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frcxn\" (UniqueName: \"kubernetes.io/projected/f5938821-854c-426f-8023-66b141804fe7-kube-api-access-frcxn\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:00.488987 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:00.488952 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:00.489143 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:00.489022 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:00.491456 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:00.491420 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:00.491456 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:00.491437 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs\") pod \"maas-controller-5cd6fb4d4f-sn2fc\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:00.773566 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:00.773471 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:01.053746 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:01.053662 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cd6fb4d4f-sn2fc"] Sep 10 04:50:01.057557 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:50:01.057529 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5938821_854c_426f_8023_66b141804fe7.slice/crio-5f5716365036240d56e12dd2138de72309044740300e70e3ca48d24c4fafa9c9 WatchSource:0}: Error finding container 5f5716365036240d56e12dd2138de72309044740300e70e3ca48d24c4fafa9c9: Status 404 returned error can't find the container with id 5f5716365036240d56e12dd2138de72309044740300e70e3ca48d24c4fafa9c9 Sep 10 04:50:01.702007 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:01.701969 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" event={"ID":"f5938821-854c-426f-8023-66b141804fe7","Type":"ContainerStarted","Data":"5f5716365036240d56e12dd2138de72309044740300e70e3ca48d24c4fafa9c9"} Sep 10 04:50:03.713856 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:03.713820 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" event={"ID":"f5938821-854c-426f-8023-66b141804fe7","Type":"ContainerStarted","Data":"8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451"} Sep 10 04:50:03.714200 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:03.713963 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:03.732708 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:03.732643 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" podStartSLOduration=2.249428746 podStartE2EDuration="4.732628818s" totalImagesPullingTime="2.483200072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:50:03.732112281 +0000 UTC m=+529.827579044" watchObservedRunningTime="2026-09-10 04:50:03.732628818 +0000 UTC m=+529.828095549" Sep 10 04:50:14.720971 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:14.720941 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:14.945536 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:14.945506 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cd6fb4d4f-sn2fc"] Sep 10 04:50:14.945752 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:14.945718 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" podUID="f5938821-854c-426f-8023-66b141804fe7" containerName="manager" containerID="cri-o://8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451" gracePeriod=30 Sep 10 04:50:15.181960 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.181936 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:15.318819 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.318736 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs\") pod \"f5938821-854c-426f-8023-66b141804fe7\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " Sep 10 04:50:15.318819 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.318773 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert\") pod \"f5938821-854c-426f-8023-66b141804fe7\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " Sep 10 04:50:15.318819 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.318807 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frcxn\" (UniqueName: \"kubernetes.io/projected/f5938821-854c-426f-8023-66b141804fe7-kube-api-access-frcxn\") pod \"f5938821-854c-426f-8023-66b141804fe7\" (UID: \"f5938821-854c-426f-8023-66b141804fe7\") " Sep 10 04:50:15.321104 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.321073 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f5938821-854c-426f-8023-66b141804fe7" (UID: "f5938821-854c-426f-8023-66b141804fe7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:50:15.321231 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.321164 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f5938821-854c-426f-8023-66b141804fe7" (UID: "f5938821-854c-426f-8023-66b141804fe7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:50:15.321291 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.321234 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5938821-854c-426f-8023-66b141804fe7-kube-api-access-frcxn" (OuterVolumeSpecName: "kube-api-access-frcxn") pod "f5938821-854c-426f-8023-66b141804fe7" (UID: "f5938821-854c-426f-8023-66b141804fe7"). InnerVolumeSpecName "kube-api-access-frcxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:50:15.419731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.419705 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-metrics-certs\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:50:15.419731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.419728 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5938821-854c-426f-8023-66b141804fe7-webhook-cert\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:50:15.419731 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.419736 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frcxn\" (UniqueName: \"kubernetes.io/projected/f5938821-854c-426f-8023-66b141804fe7-kube-api-access-frcxn\") on node \"ip-10-0-136-230.ec2.internal\" DevicePath \"\"" Sep 10 04:50:15.768134 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.768097 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5938821-854c-426f-8023-66b141804fe7" containerID="8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451" exitCode=0 Sep 10 04:50:15.768648 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.768142 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" Sep 10 04:50:15.768648 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.768171 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" event={"ID":"f5938821-854c-426f-8023-66b141804fe7","Type":"ContainerDied","Data":"8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451"} Sep 10 04:50:15.768648 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.768211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cd6fb4d4f-sn2fc" event={"ID":"f5938821-854c-426f-8023-66b141804fe7","Type":"ContainerDied","Data":"5f5716365036240d56e12dd2138de72309044740300e70e3ca48d24c4fafa9c9"} Sep 10 04:50:15.768648 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.768231 2565 scope.go:117] "RemoveContainer" containerID="8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451" Sep 10 04:50:15.778307 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.778291 2565 scope.go:117] "RemoveContainer" containerID="8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451" Sep 10 04:50:15.778571 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:50:15.778552 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451\": container with ID starting with 8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451 not found: ID does not exist" containerID="8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451" Sep 10 04:50:15.778640 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.778578 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451"} err="failed to get container status \"8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451\": rpc error: code = NotFound desc = could not find container \"8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451\": container with ID starting with 8a3cbea34d701ba57ae3044b02d74243286d3f7c5fbd7aab07cf43db503ce451 not found: ID does not exist" Sep 10 04:50:15.789333 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.789311 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cd6fb4d4f-sn2fc"] Sep 10 04:50:15.792791 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:15.792764 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5cd6fb4d4f-sn2fc"] Sep 10 04:50:16.384812 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:16.384774 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5938821-854c-426f-8023-66b141804fe7" path="/var/lib/kubelet/pods/f5938821-854c-426f-8023-66b141804fe7/volumes" Sep 10 04:50:37.263510 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.263476 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm"] Sep 10 04:50:37.264000 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.263800 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5938821-854c-426f-8023-66b141804fe7" containerName="manager" Sep 10 04:50:37.264000 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.263812 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5938821-854c-426f-8023-66b141804fe7" containerName="manager" Sep 10 04:50:37.264000 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.263864 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5938821-854c-426f-8023-66b141804fe7" containerName="manager" Sep 10 04:50:37.268689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.268672 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.271261 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.271243 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-z7848\"" Sep 10 04:50:37.271346 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.271322 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 04:50:37.275437 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.275408 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm"] Sep 10 04:50:37.388188 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.388157 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:50:37.395853 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.395828 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj2wr\" (UniqueName: \"kubernetes.io/projected/597bfb23-fbc9-43e3-997f-1e19025f0518-kube-api-access-zj2wr\") pod \"payload-pre-processing-59fbc47bf8-gv4qm\" (UID: \"597bfb23-fbc9-43e3-997f-1e19025f0518\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.395973 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.395885 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/597bfb23-fbc9-43e3-997f-1e19025f0518-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gv4qm\" (UID: \"597bfb23-fbc9-43e3-997f-1e19025f0518\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.497192 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.497153 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj2wr\" (UniqueName: \"kubernetes.io/projected/597bfb23-fbc9-43e3-997f-1e19025f0518-kube-api-access-zj2wr\") pod \"payload-pre-processing-59fbc47bf8-gv4qm\" (UID: \"597bfb23-fbc9-43e3-997f-1e19025f0518\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.497325 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.497236 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/597bfb23-fbc9-43e3-997f-1e19025f0518-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gv4qm\" (UID: \"597bfb23-fbc9-43e3-997f-1e19025f0518\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.497990 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.497970 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/597bfb23-fbc9-43e3-997f-1e19025f0518-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gv4qm\" (UID: \"597bfb23-fbc9-43e3-997f-1e19025f0518\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.507088 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.507065 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj2wr\" (UniqueName: \"kubernetes.io/projected/597bfb23-fbc9-43e3-997f-1e19025f0518-kube-api-access-zj2wr\") pod \"payload-pre-processing-59fbc47bf8-gv4qm\" (UID: \"597bfb23-fbc9-43e3-997f-1e19025f0518\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.580232 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.580162 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:37.848011 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.847989 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm"] Sep 10 04:50:37.850175 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:50:37.850147 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597bfb23_fbc9_43e3_997f_1e19025f0518.slice/crio-ff3c29346aa24f178527c29bd0cc39d9092d4e19de62209ba535b2c1747c9a5d WatchSource:0}: Error finding container ff3c29346aa24f178527c29bd0cc39d9092d4e19de62209ba535b2c1747c9a5d: Status 404 returned error can't find the container with id ff3c29346aa24f178527c29bd0cc39d9092d4e19de62209ba535b2c1747c9a5d Sep 10 04:50:37.866122 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:37.866087 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" event={"ID":"597bfb23-fbc9-43e3-997f-1e19025f0518","Type":"ContainerStarted","Data":"ff3c29346aa24f178527c29bd0cc39d9092d4e19de62209ba535b2c1747c9a5d"} Sep 10 04:50:39.878124 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:39.878086 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" event={"ID":"597bfb23-fbc9-43e3-997f-1e19025f0518","Type":"ContainerStarted","Data":"6b43eb82a346fe07a6851ef6eb828e5054ef622f84b5f30e51b6b707d6bf4f62"} Sep 10 04:50:39.878631 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:39.878195 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:39.894578 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:39.894532 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" podStartSLOduration=1.036228569 podStartE2EDuration="2.89451901s" totalImagesPullingTime="1.858290441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:50:39.893760688 +0000 UTC m=+565.989227448" watchObservedRunningTime="2026-09-10 04:50:39.89451901 +0000 UTC m=+565.989985746" Sep 10 04:50:50.885489 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:50.885456 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gv4qm" Sep 10 04:50:51.696254 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:50:51.696224 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:51:10.597907 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:10.597871 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:51:26.192972 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:26.192942 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:51:51.789797 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.789721 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h"] Sep 10 04:51:51.793152 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.793131 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:51.796667 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.796638 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dx48m\"" Sep 10 04:51:51.796667 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.796663 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 04:51:51.796874 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.796668 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 10 04:51:51.796988 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.796970 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 04:51:51.802415 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.802391 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h"] Sep 10 04:51:51.911587 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.911545 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62083c7e-a3b9-4f01-94e4-f5c94428db80-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:51.911783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.911596 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:51.911783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.911663 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:51.911783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.911681 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmql\" (UniqueName: \"kubernetes.io/projected/62083c7e-a3b9-4f01-94e4-f5c94428db80-kube-api-access-9zmql\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:51.911783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.911716 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:51.911783 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:51.911772 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.012939 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.012894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62083c7e-a3b9-4f01-94e4-f5c94428db80-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013108 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.012949 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013108 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.012970 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013108 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.013089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmql\" (UniqueName: \"kubernetes.io/projected/62083c7e-a3b9-4f01-94e4-f5c94428db80-kube-api-access-9zmql\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013256 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.013127 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013256 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.013168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013356 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.013297 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013408 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.013356 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.013456 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.013421 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.015259 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.015234 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62083c7e-a3b9-4f01-94e4-f5c94428db80-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.015521 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.015492 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62083c7e-a3b9-4f01-94e4-f5c94428db80-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.020998 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.020977 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmql\" (UniqueName: \"kubernetes.io/projected/62083c7e-a3b9-4f01-94e4-f5c94428db80-kube-api-access-9zmql\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d6r2h\" (UID: \"62083c7e-a3b9-4f01-94e4-f5c94428db80\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.104265 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.104182 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:51:52.373233 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.373210 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h"] Sep 10 04:51:52.375460 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:51:52.375434 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62083c7e_a3b9_4f01_94e4_f5c94428db80.slice/crio-91ea8049bbe1f8a67defd52dda2e0397799bbb8d4ae6f827298d1c5556d82200 WatchSource:0}: Error finding container 91ea8049bbe1f8a67defd52dda2e0397799bbb8d4ae6f827298d1c5556d82200: Status 404 returned error can't find the container with id 91ea8049bbe1f8a67defd52dda2e0397799bbb8d4ae6f827298d1c5556d82200 Sep 10 04:51:52.376843 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.376827 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 04:51:52.989547 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:52.989462 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:51:53.195669 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:53.195625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" event={"ID":"62083c7e-a3b9-4f01-94e4-f5c94428db80","Type":"ContainerStarted","Data":"91ea8049bbe1f8a67defd52dda2e0397799bbb8d4ae6f827298d1c5556d82200"} Sep 10 04:51:54.202237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:54.202199 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" event={"ID":"62083c7e-a3b9-4f01-94e4-f5c94428db80","Type":"ContainerStarted","Data":"d4c1035eaf93e666960352c13e832b8542619b4fac28fc630e6c2c36f1b5a954"} Sep 10 04:51:54.219701 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:51:54.219648 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" podStartSLOduration=2.211878906 podStartE2EDuration="3.219632871s" totalImagesPullingTime="1.007753965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:51:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:51:54.21771243 +0000 UTC m=+640.313179179" watchObservedRunningTime="2026-09-10 04:51:54.219632871 +0000 UTC m=+640.315099608" Sep 10 04:52:00.285826 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:00.285790 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:52:02.104468 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:02.104431 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:52:02.104884 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:02.104743 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:52:02.117209 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:02.117186 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:52:02.248637 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:02.248595 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d6r2h" Sep 10 04:52:21.085040 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.085005 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh"] Sep 10 04:52:21.088116 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.088094 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.090521 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.090500 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 04:52:21.096278 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.096252 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh"] Sep 10 04:52:21.269589 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.269555 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5d6l\" (UniqueName: \"kubernetes.io/projected/ed1bca28-aacd-40f7-878d-e11f779925f4-kube-api-access-j5d6l\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.269589 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.269594 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.269883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.269699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.269883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.269792 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed1bca28-aacd-40f7-878d-e11f779925f4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.269883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.269844 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.269883 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.269866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.370866 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.370768 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.370866 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.370827 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.371109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.370876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed1bca28-aacd-40f7-878d-e11f779925f4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.371109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.370900 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.371109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.370920 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.371109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.370973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5d6l\" (UniqueName: \"kubernetes.io/projected/ed1bca28-aacd-40f7-878d-e11f779925f4-kube-api-access-j5d6l\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.371313 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.371254 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.371313 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.371286 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.371382 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.371333 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.373236 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.373210 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed1bca28-aacd-40f7-878d-e11f779925f4-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.373425 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.373405 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed1bca28-aacd-40f7-878d-e11f779925f4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.378721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.378701 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5d6l\" (UniqueName: \"kubernetes.io/projected/ed1bca28-aacd-40f7-878d-e11f779925f4-kube-api-access-j5d6l\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh\" (UID: \"ed1bca28-aacd-40f7-878d-e11f779925f4\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.399419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.399394 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:21.666707 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:21.666680 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh"] Sep 10 04:52:21.669816 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:52:21.669787 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1bca28_aacd_40f7_878d_e11f779925f4.slice/crio-54cd1f602b2427353082c4a5d24b1a5e84e594730701d592ddc2d827c3a1b015 WatchSource:0}: Error finding container 54cd1f602b2427353082c4a5d24b1a5e84e594730701d592ddc2d827c3a1b015: Status 404 returned error can't find the container with id 54cd1f602b2427353082c4a5d24b1a5e84e594730701d592ddc2d827c3a1b015 Sep 10 04:52:22.324860 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:22.324763 2565 generic.go:356] "Generic (PLEG): container finished" podID="ed1bca28-aacd-40f7-878d-e11f779925f4" containerID="5eb481e1e30a6193f9d286ab807cb8f7788c9be71fb84053127e3dfabd635200" exitCode=0 Sep 10 04:52:22.324860 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:22.324845 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" event={"ID":"ed1bca28-aacd-40f7-878d-e11f779925f4","Type":"ContainerDied","Data":"5eb481e1e30a6193f9d286ab807cb8f7788c9be71fb84053127e3dfabd635200"} Sep 10 04:52:22.325290 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:22.324891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" event={"ID":"ed1bca28-aacd-40f7-878d-e11f779925f4","Type":"ContainerStarted","Data":"54cd1f602b2427353082c4a5d24b1a5e84e594730701d592ddc2d827c3a1b015"} Sep 10 04:52:22.325290 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:22.325115 2565 scope.go:117] "RemoveContainer" containerID="5eb481e1e30a6193f9d286ab807cb8f7788c9be71fb84053127e3dfabd635200" Sep 10 04:52:23.090901 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:23.090862 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:52:23.331621 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:23.331580 2565 generic.go:356] "Generic (PLEG): container finished" podID="ed1bca28-aacd-40f7-878d-e11f779925f4" containerID="bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91" exitCode=0 Sep 10 04:52:23.332053 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:23.331639 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" event={"ID":"ed1bca28-aacd-40f7-878d-e11f779925f4","Type":"ContainerDied","Data":"bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91"} Sep 10 04:52:23.332053 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:23.331698 2565 scope.go:117] "RemoveContainer" containerID="5eb481e1e30a6193f9d286ab807cb8f7788c9be71fb84053127e3dfabd635200" Sep 10 04:52:23.332053 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:23.331950 2565 scope.go:117] "RemoveContainer" containerID="bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91" Sep 10 04:52:23.332185 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:23.332141 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:52:24.338273 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:24.338237 2565 scope.go:117] "RemoveContainer" containerID="bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91" Sep 10 04:52:24.338693 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:24.338405 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:52:31.399566 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:31.399531 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:31.400054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:31.399597 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:31.400054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:31.399643 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:31.400054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:31.399665 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:31.400054 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:31.400000 2565 scope.go:117] "RemoveContainer" containerID="bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91" Sep 10 04:52:31.400198 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:31.400162 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:52:32.372911 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:32.372881 2565 scope.go:117] "RemoveContainer" containerID="bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91" Sep 10 04:52:32.373081 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:32.373048 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:52:33.202237 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.202204 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz"] Sep 10 04:52:33.206913 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.206892 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.209430 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.209411 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 10 04:52:33.217420 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.217400 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz"] Sep 10 04:52:33.270250 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.270218 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.270412 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.270302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6qkc\" (UniqueName: \"kubernetes.io/projected/eb2bbb14-b6f0-46ea-be42-3b83037343c6-kube-api-access-w6qkc\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.270412 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.270373 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.270412 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.270408 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.270588 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.270443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb2bbb14-b6f0-46ea-be42-3b83037343c6-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.270588 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.270512 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.371688 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.371660 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.371867 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.371695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.371867 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.371719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb2bbb14-b6f0-46ea-be42-3b83037343c6-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.371867 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.371754 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.371867 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.371798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.371867 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.371820 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6qkc\" (UniqueName: \"kubernetes.io/projected/eb2bbb14-b6f0-46ea-be42-3b83037343c6-kube-api-access-w6qkc\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.372148 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.372083 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.372148 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.372129 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.372224 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.372204 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.374239 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.374208 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb2bbb14-b6f0-46ea-be42-3b83037343c6-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.374339 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.374277 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb2bbb14-b6f0-46ea-be42-3b83037343c6-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.379095 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.379069 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6qkc\" (UniqueName: \"kubernetes.io/projected/eb2bbb14-b6f0-46ea-be42-3b83037343c6-kube-api-access-w6qkc\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz\" (UID: \"eb2bbb14-b6f0-46ea-be42-3b83037343c6\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.515996 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.515906 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:33.788039 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:33.787956 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz"] Sep 10 04:52:33.791186 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:52:33.791156 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb2bbb14_b6f0_46ea_be42_3b83037343c6.slice/crio-780edc60cf8f30cdf528f4cfa44589f0d51f2bb2c241356715f5951ce8236e39 WatchSource:0}: Error finding container 780edc60cf8f30cdf528f4cfa44589f0d51f2bb2c241356715f5951ce8236e39: Status 404 returned error can't find the container with id 780edc60cf8f30cdf528f4cfa44589f0d51f2bb2c241356715f5951ce8236e39 Sep 10 04:52:34.384125 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:34.384016 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" containerID="58006df60ee10d920296e2e5e4467760b363bb5d2200fc4432e8aca44fbee7a6" exitCode=0 Sep 10 04:52:34.384543 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:34.384366 2565 scope.go:117] "RemoveContainer" containerID="58006df60ee10d920296e2e5e4467760b363bb5d2200fc4432e8aca44fbee7a6" Sep 10 04:52:34.386085 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:34.385998 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" event={"ID":"eb2bbb14-b6f0-46ea-be42-3b83037343c6","Type":"ContainerDied","Data":"58006df60ee10d920296e2e5e4467760b363bb5d2200fc4432e8aca44fbee7a6"} Sep 10 04:52:34.386085 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:34.386040 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" event={"ID":"eb2bbb14-b6f0-46ea-be42-3b83037343c6","Type":"ContainerStarted","Data":"780edc60cf8f30cdf528f4cfa44589f0d51f2bb2c241356715f5951ce8236e39"} Sep 10 04:52:35.390431 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:35.390398 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" containerID="f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2" exitCode=0 Sep 10 04:52:35.390880 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:35.390487 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" event={"ID":"eb2bbb14-b6f0-46ea-be42-3b83037343c6","Type":"ContainerDied","Data":"f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2"} Sep 10 04:52:35.390880 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:35.390535 2565 scope.go:117] "RemoveContainer" containerID="58006df60ee10d920296e2e5e4467760b363bb5d2200fc4432e8aca44fbee7a6" Sep 10 04:52:35.390880 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:35.390779 2565 scope.go:117] "RemoveContainer" containerID="f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2" Sep 10 04:52:35.391028 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:35.390930 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:52:35.684150 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:35.684113 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cv7r9"] Sep 10 04:52:36.396484 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:36.396452 2565 scope.go:117] "RemoveContainer" containerID="f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2" Sep 10 04:52:36.396889 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:36.396624 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:52:43.516867 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:43.516834 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:43.516867 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:43.516876 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:43.517280 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:43.517069 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:43.517280 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:43.517104 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:52:43.517280 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:43.517187 2565 scope.go:117] "RemoveContainer" containerID="f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2" Sep 10 04:52:43.517375 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:43.517340 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:52:44.429678 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:44.429651 2565 scope.go:117] "RemoveContainer" containerID="f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2" Sep 10 04:52:44.429871 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:44.429852 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:52:47.379489 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:47.379457 2565 scope.go:117] "RemoveContainer" containerID="bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91" Sep 10 04:52:48.448983 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:48.448950 2565 generic.go:356] "Generic (PLEG): container finished" podID="ed1bca28-aacd-40f7-878d-e11f779925f4" containerID="6eed9fb032438b06e86285e76e34ca8a767e2448233b21bd30e9c8622fb7d591" exitCode=0 Sep 10 04:52:48.449360 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:48.449008 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" event={"ID":"ed1bca28-aacd-40f7-878d-e11f779925f4","Type":"ContainerDied","Data":"6eed9fb032438b06e86285e76e34ca8a767e2448233b21bd30e9c8622fb7d591"} Sep 10 04:52:48.449360 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:48.449053 2565 scope.go:117] "RemoveContainer" containerID="bab99ca325910dcaebd50072630877ee46582e7669a54e7eb0c3352a85fe8d91" Sep 10 04:52:48.449447 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:48.449379 2565 scope.go:117] "RemoveContainer" containerID="6eed9fb032438b06e86285e76e34ca8a767e2448233b21bd30e9c8622fb7d591" Sep 10 04:52:48.449645 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:48.449594 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:52:51.400190 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:51.400148 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:51.400636 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:51.400214 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:51.400636 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:51.400231 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:51.400636 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:51.400245 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:52:51.400636 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:51.400577 2565 scope.go:117] "RemoveContainer" containerID="6eed9fb032438b06e86285e76e34ca8a767e2448233b21bd30e9c8622fb7d591" Sep 10 04:52:51.400835 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:52:51.400770 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:52:59.380041 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:52:59.380009 2565 scope.go:117] "RemoveContainer" containerID="f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2" Sep 10 04:53:00.502672 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:00.502638 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" containerID="d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75" exitCode=0 Sep 10 04:53:00.503120 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:00.502708 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" event={"ID":"eb2bbb14-b6f0-46ea-be42-3b83037343c6","Type":"ContainerDied","Data":"d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75"} Sep 10 04:53:00.503120 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:00.502756 2565 scope.go:117] "RemoveContainer" containerID="f462030c619689d0d79e14b87ee509bb1317ea278e958db6fe7fa710e3ba14a2" Sep 10 04:53:00.503120 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:00.503087 2565 scope.go:117] "RemoveContainer" containerID="d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75" Sep 10 04:53:00.503284 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:00.503261 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:53:03.516411 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:03.516378 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:03.516898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:03.516428 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:03.516898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:03.516465 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:03.516898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:03.516500 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:03.516898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:03.516776 2565 scope.go:117] "RemoveContainer" containerID="d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75" Sep 10 04:53:03.517047 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:03.516939 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:53:04.521866 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:04.521839 2565 scope.go:117] "RemoveContainer" containerID="d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75" Sep 10 04:53:04.522242 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:04.521990 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:53:06.380062 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:06.380030 2565 scope.go:117] "RemoveContainer" containerID="6eed9fb032438b06e86285e76e34ca8a767e2448233b21bd30e9c8622fb7d591" Sep 10 04:53:06.380521 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:06.380241 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:53:16.380139 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:16.380050 2565 scope.go:117] "RemoveContainer" containerID="d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75" Sep 10 04:53:16.380572 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:16.380254 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:53:17.379774 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:17.379737 2565 scope.go:117] "RemoveContainer" containerID="6eed9fb032438b06e86285e76e34ca8a767e2448233b21bd30e9c8622fb7d591" Sep 10 04:53:18.583909 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:18.583873 2565 generic.go:356] "Generic (PLEG): container finished" podID="ed1bca28-aacd-40f7-878d-e11f779925f4" containerID="acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc" exitCode=0 Sep 10 04:53:18.584329 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:18.583943 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" event={"ID":"ed1bca28-aacd-40f7-878d-e11f779925f4","Type":"ContainerDied","Data":"acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc"} Sep 10 04:53:18.584329 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:18.584001 2565 scope.go:117] "RemoveContainer" containerID="6eed9fb032438b06e86285e76e34ca8a767e2448233b21bd30e9c8622fb7d591" Sep 10 04:53:18.584329 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:18.584284 2565 scope.go:117] "RemoveContainer" containerID="acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc" Sep 10 04:53:18.584523 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:18.584502 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:53:21.399990 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:21.399942 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:53:21.400382 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:21.400015 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:53:21.400382 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:21.400032 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:53:21.400382 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:21.400046 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:53:21.400486 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:21.400385 2565 scope.go:117] "RemoveContainer" containerID="acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc" Sep 10 04:53:21.400561 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:21.400542 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:53:31.380002 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:31.379960 2565 scope.go:117] "RemoveContainer" containerID="d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75" Sep 10 04:53:31.654296 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:31.654272 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" containerID="c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601" exitCode=0 Sep 10 04:53:31.654400 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:31.654347 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" event={"ID":"eb2bbb14-b6f0-46ea-be42-3b83037343c6","Type":"ContainerDied","Data":"c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601"} Sep 10 04:53:31.654445 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:31.654400 2565 scope.go:117] "RemoveContainer" containerID="d31aa7a797d9fa49cf54d374444221dbd0f280bf3ce6a33fe488323f78e9af75" Sep 10 04:53:31.654749 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:31.654731 2565 scope.go:117] "RemoveContainer" containerID="c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601" Sep 10 04:53:31.654953 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:31.654931 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:53:33.516441 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:33.516405 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:33.516955 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:33.516458 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:33.516955 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:33.516474 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:33.516955 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:33.516487 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:53:33.516955 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:33.516894 2565 scope.go:117] "RemoveContainer" containerID="c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601" Sep 10 04:53:33.517160 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:33.517094 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:53:35.380010 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:35.379980 2565 scope.go:117] "RemoveContainer" containerID="acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc" Sep 10 04:53:35.380578 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:35.380223 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:53:48.379862 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:48.379830 2565 scope.go:117] "RemoveContainer" containerID="c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601" Sep 10 04:53:48.380225 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:48.380004 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:53:50.379813 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:53:50.379780 2565 scope.go:117] "RemoveContainer" containerID="acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc" Sep 10 04:53:50.380177 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:53:50.379969 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:54:02.379662 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:02.379630 2565 scope.go:117] "RemoveContainer" containerID="c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601" Sep 10 04:54:02.380072 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:02.379803 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:54:04.381643 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:04.381595 2565 scope.go:117] "RemoveContainer" containerID="acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc" Sep 10 04:54:04.808484 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:04.808448 2565 generic.go:356] "Generic (PLEG): container finished" podID="ed1bca28-aacd-40f7-878d-e11f779925f4" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" exitCode=0 Sep 10 04:54:04.808670 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:04.808488 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" event={"ID":"ed1bca28-aacd-40f7-878d-e11f779925f4","Type":"ContainerDied","Data":"4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a"} Sep 10 04:54:04.808670 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:04.808564 2565 scope.go:117] "RemoveContainer" containerID="acffab52c016b9a46774b2ad04c45eee8ecd3c81a6808579388cfd7409e10cfc" Sep 10 04:54:04.808924 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:04.808901 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:54:04.809199 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:04.809080 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:54:11.399495 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:11.399460 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:54:11.400096 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:11.399506 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:54:11.400096 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:11.399521 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:54:11.400096 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:11.399565 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:54:11.400096 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:11.399831 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:54:11.400096 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:11.400008 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:54:11.841543 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:11.841515 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:54:11.841722 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:11.841705 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:54:15.379788 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:15.379756 2565 scope.go:117] "RemoveContainer" containerID="c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601" Sep 10 04:54:15.859703 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:15.859669 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" exitCode=0 Sep 10 04:54:15.859877 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:15.859709 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" event={"ID":"eb2bbb14-b6f0-46ea-be42-3b83037343c6","Type":"ContainerDied","Data":"a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32"} Sep 10 04:54:15.859877 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:15.859745 2565 scope.go:117] "RemoveContainer" containerID="c5cc41acbb6aac8bb29b61a29e0b3e8a66bbd76e7884b9843fb47baa229c6601" Sep 10 04:54:15.860365 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:15.860136 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:54:15.860436 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:15.860377 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:54:23.379842 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:23.379802 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:54:23.380305 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:23.380046 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:54:23.516242 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:23.516210 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:54:23.516419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:23.516258 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:54:23.516419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:23.516274 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:54:23.516419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:23.516288 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:54:23.516694 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:23.516671 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:54:23.516892 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:23.516870 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:54:36.380409 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:36.380371 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:54:36.380860 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:36.380640 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:54:39.380244 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:39.380174 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:54:39.380594 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:39.380319 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:54:50.379624 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:50.379567 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:54:50.380179 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:50.379812 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:54:51.379663 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:54:51.379634 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:54:51.380020 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:54:51.379803 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:55:01.380143 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:01.380106 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:55:01.380630 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:01.380299 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:55:03.379619 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:03.379576 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:55:03.379982 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:03.379789 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:55:12.379619 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:12.379586 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:55:12.380001 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:12.379780 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:55:17.380224 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:17.380187 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:55:17.380684 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:17.380358 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:55:23.380175 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:23.380140 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:55:23.380741 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:23.380336 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:55:31.379367 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:31.379336 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:55:32.198460 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:32.198430 2565 generic.go:356] "Generic (PLEG): container finished" podID="ed1bca28-aacd-40f7-878d-e11f779925f4" containerID="6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5" exitCode=0 Sep 10 04:55:32.198681 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:32.198504 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" event={"ID":"ed1bca28-aacd-40f7-878d-e11f779925f4","Type":"ContainerDied","Data":"6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5"} Sep 10 04:55:32.198681 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:32.198578 2565 scope.go:117] "RemoveContainer" containerID="4bba0f72b7b8857ed320c9823623be756c45a0399b84903e0ae4a676c82ba22a" Sep 10 04:55:32.198940 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:32.198922 2565 scope.go:117] "RemoveContainer" containerID="6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5" Sep 10 04:55:32.199133 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:32.199114 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:55:36.379568 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:36.379537 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:55:37.223546 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:37.223514 2565 generic.go:356] "Generic (PLEG): container finished" podID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" containerID="11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9" exitCode=0 Sep 10 04:55:37.223721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:37.223587 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" event={"ID":"eb2bbb14-b6f0-46ea-be42-3b83037343c6","Type":"ContainerDied","Data":"11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9"} Sep 10 04:55:37.223721 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:37.223655 2565 scope.go:117] "RemoveContainer" containerID="a80008ddc192bddb995fa0d2ae37d7eb2e4598c264ce7d8ddbb51ea885304e32" Sep 10 04:55:37.223972 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:37.223954 2565 scope.go:117] "RemoveContainer" containerID="11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9" Sep 10 04:55:37.224139 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:37.224118 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:55:41.399764 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:41.399729 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:55:41.400163 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:41.399787 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:55:41.400163 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:41.399802 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:55:41.400163 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:41.399816 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" Sep 10 04:55:41.400163 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:41.400123 2565 scope.go:117] "RemoveContainer" containerID="6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5" Sep 10 04:55:41.400297 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:41.400280 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:55:43.516518 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:43.516479 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:55:43.516518 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:43.516528 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:55:43.516999 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:43.516539 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:55:43.516999 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:43.516548 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" Sep 10 04:55:43.516999 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:43.516894 2565 scope.go:117] "RemoveContainer" containerID="11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9" Sep 10 04:55:43.517100 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:43.517046 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:55:49.997133 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:49.997102 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_ed1bca28-aacd-40f7-878d-e11f779925f4/main/5.log" Sep 10 04:55:50.111231 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:50.111198 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_ed1bca28-aacd-40f7-878d-e11f779925f4/main/5.log" Sep 10 04:55:56.379844 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:56.379809 2565 scope.go:117] "RemoveContainer" containerID="6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5" Sep 10 04:55:56.380397 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:56.380040 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:55:56.958965 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:56.958934 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6766b6589c-dvvkh_6e0adb95-b2fe-48f2-8dcb-9699a8214fa0/copy-manifests/0.log" Sep 10 04:55:56.971114 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:56.971084 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6766b6589c-dvvkh_6e0adb95-b2fe-48f2-8dcb-9699a8214fa0/manager/0.log" Sep 10 04:55:57.379879 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:57.379802 2565 scope.go:117] "RemoveContainer" containerID="11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9" Sep 10 04:55:57.380220 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:55:57.379959 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:55:58.594937 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:58.594906 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-jcpk7_3ab35c66-2f40-47a6-8a81-67b3a81e17ba/manager/0.log" Sep 10 04:55:58.923188 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:58.923157 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-cv7r9_7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f/limitador/0.log" Sep 10 04:55:59.728238 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:59.728206 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbb8ff44d-m9l5z_46b7b3fd-811e-4ca3-be0d-346acd7a5490/kube-auth-proxy/0.log" Sep 10 04:55:59.952671 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:55:59.952640 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gv4qm_597bfb23-fbc9-43e3-997f-1e19025f0518/payload-pre-processing/0.log" Sep 10 04:56:00.838211 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:00.838180 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-d6r2h_62083c7e-a3b9-4f01-94e4-f5c94428db80/main/0.log" Sep 10 04:56:01.059691 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:01.059661 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_ed1bca28-aacd-40f7-878d-e11f779925f4/main/5.log" Sep 10 04:56:01.169692 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:01.169668 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_eb2bbb14-b6f0-46ea-be42-3b83037343c6/main/5.log" Sep 10 04:56:07.379300 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:07.379267 2565 scope.go:117] "RemoveContainer" containerID="6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5" Sep 10 04:56:07.379788 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:56:07.379493 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:56:09.278743 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:09.278709 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f2xkr_f065d09c-3151-4335-bf0d-05452ac3aa70/global-pull-secret-syncer/0.log" Sep 10 04:56:09.402397 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:09.402366 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vhws4_2b82cf9f-b435-4bd4-bd01-bbba2dcdd6a3/konnectivity-agent/0.log" Sep 10 04:56:09.491461 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:09.491430 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-230.ec2.internal_d97b5ec804f662bb91372d0961fb0075/haproxy/0.log" Sep 10 04:56:10.380333 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:10.380307 2565 scope.go:117] "RemoveContainer" containerID="11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9" Sep 10 04:56:10.380715 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:56:10.380451 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:56:13.940538 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:13.940510 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-jcpk7_3ab35c66-2f40-47a6-8a81-67b3a81e17ba/manager/0.log" Sep 10 04:56:14.044105 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:14.044075 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-cv7r9_7b8c5fd5-3ef7-4034-ba9c-9a73ddf2905f/limitador/0.log" Sep 10 04:56:15.923889 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:15.923858 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqwnv_b65fc1a8-c522-453e-82e5-8d1997135aba/kube-state-metrics/0.log" Sep 10 04:56:15.942255 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:15.942228 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqwnv_b65fc1a8-c522-453e-82e5-8d1997135aba/kube-rbac-proxy-main/0.log" Sep 10 04:56:15.963764 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:15.963743 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqwnv_b65fc1a8-c522-453e-82e5-8d1997135aba/kube-rbac-proxy-self/0.log" Sep 10 04:56:15.995377 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:15.995349 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fcffb998d-69rvp_5330794d-4870-436c-8763-03237c61ffeb/metrics-server/0.log" Sep 10 04:56:16.203529 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.203460 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s9vs6_aed15e65-8bce-4e3e-a05c-b1c1e116bbc0/node-exporter/0.log" Sep 10 04:56:16.223940 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.223911 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s9vs6_aed15e65-8bce-4e3e-a05c-b1c1e116bbc0/kube-rbac-proxy/0.log" Sep 10 04:56:16.244630 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.244597 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s9vs6_aed15e65-8bce-4e3e-a05c-b1c1e116bbc0/init-textfile/0.log" Sep 10 04:56:16.274182 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.274160 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-bzjn2_3363df8b-023e-4b28-a595-8dd9f056f9da/kube-rbac-proxy-main/0.log" Sep 10 04:56:16.294861 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.294838 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-bzjn2_3363df8b-023e-4b28-a595-8dd9f056f9da/kube-rbac-proxy-self/0.log" Sep 10 04:56:16.323707 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.323688 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-bzjn2_3363df8b-023e-4b28-a595-8dd9f056f9da/openshift-state-metrics/0.log" Sep 10 04:56:16.388519 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.388496 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_145c9bfb-a936-4a1c-8d18-1fa3ab91c25e/prometheus/0.log" Sep 10 04:56:16.459298 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.459223 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_145c9bfb-a936-4a1c-8d18-1fa3ab91c25e/config-reloader/0.log" Sep 10 04:56:16.521824 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.521798 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_145c9bfb-a936-4a1c-8d18-1fa3ab91c25e/thanos-sidecar/0.log" Sep 10 04:56:16.550893 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.550867 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_145c9bfb-a936-4a1c-8d18-1fa3ab91c25e/kube-rbac-proxy-web/0.log" Sep 10 04:56:16.588125 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.588100 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_145c9bfb-a936-4a1c-8d18-1fa3ab91c25e/kube-rbac-proxy/0.log" Sep 10 04:56:16.619109 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.619082 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_145c9bfb-a936-4a1c-8d18-1fa3ab91c25e/kube-rbac-proxy-thanos/0.log" Sep 10 04:56:16.645489 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.645463 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_145c9bfb-a936-4a1c-8d18-1fa3ab91c25e/init-config-reloader/0.log" Sep 10 04:56:16.678877 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.678853 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pnvgw_05798918-4dae-42e7-8af3-b4cee9cfc2d1/prometheus-operator/0.log" Sep 10 04:56:16.699346 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.699325 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pnvgw_05798918-4dae-42e7-8af3-b4cee9cfc2d1/kube-rbac-proxy/0.log" Sep 10 04:56:16.726794 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:16.726736 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-2r7gl_5ab97a86-da91-49d2-ad05-abf5b791e314/prometheus-operator-admission-webhook/0.log" Sep 10 04:56:19.776542 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.776508 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl"] Sep 10 04:56:19.779702 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.779682 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:19.781899 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.781877 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cwtlm\"/\"default-dockercfg-9jktl\"" Sep 10 04:56:19.782830 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.782813 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cwtlm\"/\"kube-root-ca.crt\"" Sep 10 04:56:19.782830 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.782819 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cwtlm\"/\"openshift-service-ca.crt\"" Sep 10 04:56:19.788561 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.788538 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl"] Sep 10 04:56:19.934067 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.934033 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-lib-modules\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:19.934241 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.934080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksxxk\" (UniqueName: \"kubernetes.io/projected/f9e5c730-eeb7-4afb-8334-5bd17082496c-kube-api-access-ksxxk\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:19.934241 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.934100 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-proc\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:19.934241 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.934125 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-sys\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:19.934241 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:19.934157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-podres\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.034884 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.034802 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-lib-modules\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.034884 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.034850 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksxxk\" (UniqueName: \"kubernetes.io/projected/f9e5c730-eeb7-4afb-8334-5bd17082496c-kube-api-access-ksxxk\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.034884 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.034869 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-proc\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.034884 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.034887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-sys\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.035148 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.034919 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-podres\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.035148 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.034965 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-proc\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.035148 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.034987 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-lib-modules\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.035148 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.035030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-podres\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.035148 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.035030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9e5c730-eeb7-4afb-8334-5bd17082496c-sys\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.042401 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.042380 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksxxk\" (UniqueName: \"kubernetes.io/projected/f9e5c730-eeb7-4afb-8334-5bd17082496c-kube-api-access-ksxxk\") pod \"perf-node-gather-daemonset-s7fpl\" (UID: \"f9e5c730-eeb7-4afb-8334-5bd17082496c\") " pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.090502 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.090471 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:20.361457 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.361387 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl"] Sep 10 04:56:20.364647 ip-10-0-136-230 kubenswrapper[2565]: W0910 04:56:20.364593 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf9e5c730_eeb7_4afb_8334_5bd17082496c.slice/crio-23b75df29f10dfb4561714af514a1271897813959ef4b197544902cfca513619 WatchSource:0}: Error finding container 23b75df29f10dfb4561714af514a1271897813959ef4b197544902cfca513619: Status 404 returned error can't find the container with id 23b75df29f10dfb4561714af514a1271897813959ef4b197544902cfca513619 Sep 10 04:56:20.411614 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:20.411569 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" event={"ID":"f9e5c730-eeb7-4afb-8334-5bd17082496c","Type":"ContainerStarted","Data":"23b75df29f10dfb4561714af514a1271897813959ef4b197544902cfca513619"} Sep 10 04:56:21.417231 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:21.417197 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" event={"ID":"f9e5c730-eeb7-4afb-8334-5bd17082496c","Type":"ContainerStarted","Data":"bd00e36abd930a2d09a98c0beb43dc8bf0cfa7a94532bbf2c6c9243e6180899b"} Sep 10 04:56:21.417641 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:21.417341 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:21.435115 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:21.435070 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" podStartSLOduration=2.435058536 podStartE2EDuration="2.435058536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:56:21.43264116 +0000 UTC m=+907.528107919" watchObservedRunningTime="2026-09-10 04:56:21.435058536 +0000 UTC m=+907.530525272" Sep 10 04:56:21.681365 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:21.681295 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w82t5_f1f53af7-3313-491c-829d-3658b362fae2/dns/0.log" Sep 10 04:56:21.704421 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:21.704397 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w82t5_f1f53af7-3313-491c-829d-3658b362fae2/kube-rbac-proxy/0.log" Sep 10 04:56:21.777800 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:21.777770 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4n6vl_7d2881f6-4630-4769-90e6-ea5c9af3f4d8/dns-node-resolver/0.log" Sep 10 04:56:22.379567 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:22.379534 2565 scope.go:117] "RemoveContainer" containerID="6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5" Sep 10 04:56:22.379769 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:22.379704 2565 scope.go:117] "RemoveContainer" containerID="11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9" Sep 10 04:56:22.379825 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:56:22.379786 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:56:22.379872 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:56:22.379846 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:56:22.384354 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:22.384330 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56775dcb96-hkm7j_5295dded-ad87-480c-9126-7ca5867708b0/registry/0.log" Sep 10 04:56:22.434215 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:22.434187 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bllrf_a308358a-d20b-40d7-b9bc-a95c79d0c32a/node-ca/0.log" Sep 10 04:56:23.496523 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:23.496493 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbb8ff44d-m9l5z_46b7b3fd-811e-4ca3-be0d-346acd7a5490/kube-auth-proxy/0.log" Sep 10 04:56:23.546589 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:23.546562 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gv4qm_597bfb23-fbc9-43e3-997f-1e19025f0518/payload-pre-processing/0.log" Sep 10 04:56:24.186830 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:24.186804 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-29hct_4bdee842-eefe-4d76-901f-15838d2e279b/serve-healthcheck-canary/0.log" Sep 10 04:56:24.711304 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:24.711280 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgxh2_e4896bd2-5b68-4f2e-be91-88c0c7727440/kube-rbac-proxy/0.log" Sep 10 04:56:24.730993 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:24.730972 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgxh2_e4896bd2-5b68-4f2e-be91-88c0c7727440/exporter/0.log" Sep 10 04:56:24.751542 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:24.751518 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgxh2_e4896bd2-5b68-4f2e-be91-88c0c7727440/extractor/0.log" Sep 10 04:56:27.170041 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:27.170012 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6766b6589c-dvvkh_6e0adb95-b2fe-48f2-8dcb-9699a8214fa0/manager/0.log" Sep 10 04:56:27.186419 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:27.186397 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6766b6589c-dvvkh_6e0adb95-b2fe-48f2-8dcb-9699a8214fa0/copy-manifests/0.log" Sep 10 04:56:27.431291 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:27.431222 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cwtlm/perf-node-gather-daemonset-s7fpl" Sep 10 04:56:28.492792 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:28.492758 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64cfcb4bb4-sr945_1361a7cf-1311-452f-92e7-b015d62ca293/manager/0.log" Sep 10 04:56:30.610104 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:30.610073 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbb8ff44d-m9l5z_46b7b3fd-811e-4ca3-be0d-346acd7a5490/kube-auth-proxy/0.log" Sep 10 04:56:30.666036 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:30.666002 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gv4qm_597bfb23-fbc9-43e3-997f-1e19025f0518/payload-pre-processing/0.log" Sep 10 04:56:31.937727 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:31.937703 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbb8ff44d-m9l5z_46b7b3fd-811e-4ca3-be0d-346acd7a5490/kube-auth-proxy/0.log" Sep 10 04:56:31.977073 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:31.977046 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gv4qm_597bfb23-fbc9-43e3-997f-1e19025f0518/payload-pre-processing/0.log" Sep 10 04:56:34.382251 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:34.382223 2565 scope.go:117] "RemoveContainer" containerID="11e9fde8fec55102c7f79c8bf8204501205a7629cfa8af38cd2121f201cb18b9" Sep 10 04:56:34.382654 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:56:34.382390 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz_llm(eb2bbb14-b6f0-46ea-be42-3b83037343c6)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-cp8tz" podUID="eb2bbb14-b6f0-46ea-be42-3b83037343c6" Sep 10 04:56:35.348047 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.348016 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrd8j_c417b44d-13b8-4555-83df-7e5229f14823/kube-multus-additional-cni-plugins/0.log" Sep 10 04:56:35.368973 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.368945 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrd8j_c417b44d-13b8-4555-83df-7e5229f14823/egress-router-binary-copy/0.log" Sep 10 04:56:35.379357 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.379336 2565 scope.go:117] "RemoveContainer" containerID="6616ee11d2674d4e9cdadcd6afef49ad9e9cf89063836fe88464790e073d59d5" Sep 10 04:56:35.379508 ip-10-0-136-230 kubenswrapper[2565]: E0910 04:56:35.379492 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh_llm(ed1bca28-aacd-40f7-878d-e11f779925f4)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l2rlh" podUID="ed1bca28-aacd-40f7-878d-e11f779925f4" Sep 10 04:56:35.390898 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.390881 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrd8j_c417b44d-13b8-4555-83df-7e5229f14823/cni-plugins/0.log" Sep 10 04:56:35.410982 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.410964 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrd8j_c417b44d-13b8-4555-83df-7e5229f14823/bond-cni-plugin/0.log" Sep 10 04:56:35.430931 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.430913 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrd8j_c417b44d-13b8-4555-83df-7e5229f14823/routeoverride-cni/0.log" Sep 10 04:56:35.451309 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.451287 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrd8j_c417b44d-13b8-4555-83df-7e5229f14823/whereabouts-cni-bincopy/0.log" Sep 10 04:56:35.472101 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.472083 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrd8j_c417b44d-13b8-4555-83df-7e5229f14823/whereabouts-cni/0.log" Sep 10 04:56:35.680173 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.680146 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hrctp_6f8d8b07-d521-43ac-b9cb-3e753107f87c/kube-multus/0.log" Sep 10 04:56:35.730535 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.730508 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44mnf_54664d95-d774-4a37-a7f1-eb37c2efb5ea/network-metrics-daemon/0.log" Sep 10 04:56:35.749665 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:35.749642 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44mnf_54664d95-d774-4a37-a7f1-eb37c2efb5ea/kube-rbac-proxy/0.log" Sep 10 04:56:37.023124 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.023093 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/ovn-controller/0.log" Sep 10 04:56:37.041948 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.041902 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/ovn-acl-logging/0.log" Sep 10 04:56:37.062689 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.062669 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/kube-rbac-proxy-node/0.log" Sep 10 04:56:37.082866 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.082843 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 04:56:37.102555 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.102536 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/northd/0.log" Sep 10 04:56:37.122308 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.122287 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/nbdb/0.log" Sep 10 04:56:37.144891 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.144862 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/sbdb/0.log" Sep 10 04:56:37.180739 ip-10-0-136-230 kubenswrapper[2565]: I0910 04:56:37.180713 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9cp4_ac0af446-e525-4216-adfc-6188ecc148a5/ovnkube-controller/0.log"