Sep 09 19:55:29.616895 ip-10-0-142-99 systemd[1]: Starting Kubernetes Kubelet... Sep 09 19:55:30.022867 ip-10-0-142-99 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 19:55:30.022867 ip-10-0-142-99 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 19:55:30.022867 ip-10-0-142-99 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 19:55:30.022867 ip-10-0-142-99 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 19:55:30.022867 ip-10-0-142-99 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 19:55:30.023922 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.023834 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 19:55:30.029720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029695 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 19:55:30.029720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029716 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 19:55:30.029720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029720 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 19:55:30.029720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029723 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 19:55:30.029720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029727 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029731 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029734 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029737 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029740 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029742 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029745 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029748 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029751 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029753 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029756 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029758 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029762 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029766 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029769 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029772 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029775 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029778 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029780 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029783 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 19:55:30.029923 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029789 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029792 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029794 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029797 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029800 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029802 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029805 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029807 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029810 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029812 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029815 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029817 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029820 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029823 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029827 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029831 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029834 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029837 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029839 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 19:55:30.030432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029842 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029844 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029847 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029849 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029852 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029855 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029857 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029860 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029862 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029865 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029867 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029869 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029872 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029875 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029877 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029880 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029882 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029885 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029887 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029889 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 19:55:30.030913 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029892 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029895 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029899 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029903 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029906 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029909 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029912 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029915 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029917 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029920 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029923 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029926 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029928 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029931 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029933 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029936 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029939 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029942 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029944 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 19:55:30.031407 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029947 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029950 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029953 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029955 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029958 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029960 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029962 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029965 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029968 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029970 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029974 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029976 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029979 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029982 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029984 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029986 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.029989 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030710 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030717 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030721 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 19:55:30.031885 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030724 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030726 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030729 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030732 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030734 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030737 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030740 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030744 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030746 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030749 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030752 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030754 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030757 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030759 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030762 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030765 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030768 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030771 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030773 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030776 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 19:55:30.032377 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030778 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030781 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030783 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030786 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030788 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030791 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030795 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030798 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030801 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030803 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030806 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030809 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030812 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030815 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030818 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030820 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030823 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030826 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030829 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030831 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 19:55:30.032889 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030834 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030837 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030839 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030842 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030844 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030847 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030850 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030854 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030856 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030859 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030862 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030864 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030867 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030869 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030872 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030874 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030878 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030882 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030886 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 19:55:30.033392 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030889 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030892 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030895 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030898 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030901 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030904 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030907 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030909 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030912 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030916 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030919 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030922 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030925 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030929 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030932 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030935 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030938 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030940 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030943 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030945 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 19:55:30.033858 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030948 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030951 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030954 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030956 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030959 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030961 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030964 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030966 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030969 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030972 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030975 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030977 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030980 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030982 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030985 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030987 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.030990 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031406 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031413 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031417 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 19:55:30.034358 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031420 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031423 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031427 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031431 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031434 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031437 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031440 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031444 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031447 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031449 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031452 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031454 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031457 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031462 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031464 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031467 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031469 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031472 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031474 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 19:55:30.034915 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031477 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031479 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031482 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031485 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031487 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031490 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031492 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031495 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031498 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031500 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031503 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031506 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031508 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031512 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031515 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031518 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031522 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031525 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031527 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 19:55:30.035424 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031530 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031532 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031535 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031537 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031540 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031542 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031545 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031547 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031550 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031552 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031555 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031557 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031560 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031562 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031565 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031567 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031570 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031572 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031575 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031577 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 19:55:30.035898 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031580 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031582 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031585 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031588 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031590 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031592 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031595 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031597 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031600 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031603 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031605 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031608 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031610 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031629 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031634 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031636 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031639 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031641 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031644 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031646 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 19:55:30.036379 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031649 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031654 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031656 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031659 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031662 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031664 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031666 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031671 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031674 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031676 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031679 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031681 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031684 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031686 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031689 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031692 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031694 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.031697 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031765 2577 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031801 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031807 2577 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 19:55:30.036907 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031811 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031816 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031820 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031825 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031829 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031832 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031835 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031838 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031842 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031845 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031848 2577 flags.go:64] FLAG: --cgroup-root="" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031850 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031853 2577 flags.go:64] FLAG: --client-ca-file="" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031857 2577 flags.go:64] FLAG: --cloud-provider="external" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031860 2577 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031864 2577 flags.go:64] FLAG: --cluster-domain="" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031866 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031869 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031874 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031877 2577 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031881 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031884 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031887 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 19:55:30.037405 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031890 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031893 2577 flags.go:64] FLAG: --contention-profiling="false" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031897 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031900 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031903 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031906 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031910 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031914 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031917 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031919 2577 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031922 2577 flags.go:64] FLAG: --enable-server="true" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031925 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031930 2577 flags.go:64] FLAG: --event-burst="100" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031933 2577 flags.go:64] FLAG: --event-qps="50" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031936 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031939 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031942 2577 flags.go:64] FLAG: --eviction-hard="" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031945 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031948 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031951 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031954 2577 flags.go:64] FLAG: --eviction-soft="" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031956 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031959 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031962 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031965 2577 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 19:55:30.037975 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031968 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031971 2577 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031975 2577 flags.go:64] FLAG: --feature-gates="" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031979 2577 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031982 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031985 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031988 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031990 2577 flags.go:64] FLAG: --healthz-port="10248" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031995 2577 flags.go:64] FLAG: --help="false" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.031998 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032001 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032004 2577 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032006 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032010 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032013 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032016 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032018 2577 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032021 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032024 2577 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032027 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032030 2577 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032032 2577 flags.go:64] FLAG: --kube-reserved="" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032035 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032038 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 19:55:30.038584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032041 2577 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032043 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032046 2577 flags.go:64] FLAG: --lock-file="" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032049 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032051 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032054 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032059 2577 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032062 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032065 2577 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032068 2577 flags.go:64] FLAG: --logging-format="text" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032072 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032075 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032078 2577 flags.go:64] FLAG: --manifest-url="" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032081 2577 flags.go:64] FLAG: --manifest-url-header="" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032088 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032091 2577 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032097 2577 flags.go:64] FLAG: --max-pods="110" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032100 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032102 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032105 2577 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032108 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032111 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032114 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032117 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032124 2577 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 19:55:30.039235 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032127 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032130 2577 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032133 2577 flags.go:64] FLAG: --pod-cidr="" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032136 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032140 2577 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032143 2577 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032146 2577 flags.go:64] FLAG: --pods-per-core="0" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032149 2577 flags.go:64] FLAG: --port="10250" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032152 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032155 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aa22e11886697b99" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032158 2577 flags.go:64] FLAG: --qos-reserved="" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032161 2577 flags.go:64] FLAG: --read-only-port="10255" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032164 2577 flags.go:64] FLAG: --register-node="true" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032167 2577 flags.go:64] FLAG: --register-with-taints="" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032170 2577 flags.go:64] FLAG: --registry-burst="10" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032175 2577 flags.go:64] FLAG: --registry-qps="5" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032178 2577 flags.go:64] FLAG: --reserved-cpus="" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032181 2577 flags.go:64] FLAG: --reserved-memory="" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032187 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032190 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032193 2577 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032196 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032199 2577 flags.go:64] FLAG: --runonce="false" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032203 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032206 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 19:55:30.039888 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032208 2577 flags.go:64] FLAG: --seccomp-default="false" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032211 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032214 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032217 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032220 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032223 2577 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032226 2577 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032229 2577 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032232 2577 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032234 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032237 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032240 2577 flags.go:64] FLAG: --system-cgroups="" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032243 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032248 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032251 2577 flags.go:64] FLAG: --tls-cert-file="" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032254 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032258 2577 flags.go:64] FLAG: --tls-min-version="" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032260 2577 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032263 2577 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032266 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032269 2577 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032271 2577 flags.go:64] FLAG: --v="2" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032275 2577 flags.go:64] FLAG: --version="false" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032279 2577 flags.go:64] FLAG: --vmodule="" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032283 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 19:55:30.040523 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.032288 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032403 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032407 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032410 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032413 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032417 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032420 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032423 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032425 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032428 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032431 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032433 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032436 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032439 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032441 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032444 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032446 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032448 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032451 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032453 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 19:55:30.041142 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032456 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032460 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032464 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032467 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032470 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032472 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032475 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032478 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032480 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032483 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032485 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032488 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032492 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032495 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032497 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032500 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032502 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032506 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032509 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032511 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 19:55:30.041668 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032514 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032517 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032519 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032522 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032525 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032527 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032529 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032532 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032535 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032537 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032540 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032542 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032545 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032547 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032550 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032552 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032555 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032557 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032560 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032562 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 19:55:30.042163 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032565 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032567 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032570 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032572 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032576 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032581 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032584 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032587 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032590 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032594 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032597 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032600 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032603 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032606 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032609 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032611 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032641 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032645 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032648 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032651 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 19:55:30.042662 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032653 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032656 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032658 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032661 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032663 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032666 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032669 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032671 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032673 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032676 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032679 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032681 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032683 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032687 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032691 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032693 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032697 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032700 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032703 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 19:55:30.043156 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.032705 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.033261 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.040149 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.040165 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.041139 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.041147 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041207 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041213 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041216 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041219 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041222 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041225 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041229 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041233 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 19:55:30.043637 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041237 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041240 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041243 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041246 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041249 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041252 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041255 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041257 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041260 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041262 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041265 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041268 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041270 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041273 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041276 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041278 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041282 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041285 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041288 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041290 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 19:55:30.044059 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041293 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041296 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041300 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041303 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041306 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041309 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041312 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041314 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041317 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041319 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041322 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041325 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041327 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041330 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041332 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041335 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041337 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041340 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041342 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041345 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 19:55:30.044720 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041347 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041350 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041352 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041355 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041358 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041360 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041363 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041366 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041368 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041371 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041374 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041376 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041379 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041382 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041385 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041387 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041389 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041392 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041394 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041397 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 19:55:30.045456 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041400 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041403 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041407 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041410 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041412 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041415 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041418 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041420 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041423 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041425 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041428 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041430 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041433 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041435 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041439 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041441 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041443 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041446 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041449 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 19:55:30.046180 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041452 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041454 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041456 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041459 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041462 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041465 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041468 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041470 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041473 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041475 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041478 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041480 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.041485 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041604 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 19:55:30.046680 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041609 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041612 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041632 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041636 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041639 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041642 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041645 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041648 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041651 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041653 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041656 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041659 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041661 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041664 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041669 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041672 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041676 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041679 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041682 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041684 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 19:55:30.047159 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041689 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041691 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041694 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041697 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041699 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041703 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041705 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041708 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041710 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041713 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041716 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041718 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041720 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041723 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041725 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041728 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041730 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041732 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041735 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041737 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 19:55:30.047722 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041740 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041743 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041745 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041747 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041750 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041753 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041755 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041758 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041760 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041762 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041765 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041767 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041770 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041772 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041774 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041777 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041780 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041783 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041785 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041787 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 19:55:30.048197 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041790 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041792 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041795 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041797 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041800 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041802 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041805 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041807 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041810 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041813 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041815 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041817 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041820 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041822 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041824 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041827 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041829 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041832 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041834 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 19:55:30.048700 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041837 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041839 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041842 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041844 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041848 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041851 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041853 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041856 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041858 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041861 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041864 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041866 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041869 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041871 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041874 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041876 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041879 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041881 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 19:55:30.049201 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:30.041884 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 19:55:30.049640 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.041889 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 19:55:30.049640 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.041986 2577 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 19:55:30.049640 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.043849 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 19:55:30.049640 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.044782 2577 server.go:1019] "Starting client certificate rotation" Sep 09 19:55:30.049640 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.044902 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 19:55:30.049640 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.044955 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 19:55:30.049640 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.047064 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 19:55:30.050399 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.050372 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 19:55:30.061397 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.061377 2577 log.go:25] "Validated CRI v1 runtime API" Sep 09 19:55:30.065303 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.065287 2577 log.go:25] "Validated CRI v1 image API" Sep 09 19:55:30.066250 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.066229 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 19:55:30.069534 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.069516 2577 fs.go:135] Filesystem UUIDs: map[1d86576d-c5e4-4b2c-b134-0e65170bec3a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d650aba4-4f07-4e36-8c10-6215843fa69f:/dev/nvme0n1p3] Sep 09 19:55:30.069595 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.069535 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 19:55:30.075543 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.075434 2577 manager.go:216] Machine: {Timestamp:2026-09-09 19:55:30.073066371 +0000 UTC m=+0.337403076 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094129 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec289e84970f2b1be8e7f361fbb60e65 SystemUUID:ec289e84-970f-2b1b-e8e7-f361fbb60e65 BootID:e149896b-5eb4-4224-a369-605143d41e0b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e1:68:d8:a6:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e1:68:d8:a6:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:fe:fd:2b:91:70 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 19:55:30.075543 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.075538 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 19:55:30.075687 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.075630 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 19:55:30.076810 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.076792 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 19:55:30.078477 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.078455 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 19:55:30.078652 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.078481 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 19:55:30.079181 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.079171 2577 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 19:55:30.079210 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.079183 2577 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 19:55:30.079210 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.079191 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 19:55:30.079210 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.079205 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 19:55:30.080230 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.080221 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 19:55:30.083977 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.083965 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 09 19:55:30.084090 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084081 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 19:55:30.084130 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084111 2577 kubelet.go:486] "Attempting to sync node with API server" Sep 09 19:55:30.084130 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084120 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 19:55:30.084188 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084134 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 19:55:30.084188 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084143 2577 kubelet.go:398] "Adding apiserver pod source" Sep 09 19:55:30.084188 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084152 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 19:55:30.084526 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084510 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 19:55:30.084594 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.084539 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 19:55:30.086744 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.086728 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 19:55:30.088940 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.088927 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 19:55:30.088988 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.088957 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 19:55:30.089147 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089134 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 19:55:30.089176 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089155 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 19:55:30.089176 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089164 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 19:55:30.089176 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089173 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 19:55:30.089254 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089184 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 19:55:30.089254 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089200 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 19:55:30.089660 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089651 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 19:55:30.089691 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089660 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 19:55:30.089691 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089666 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 19:55:30.089691 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089672 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 19:55:30.089691 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089685 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 19:55:30.089691 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.089692 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 19:55:30.090521 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.090511 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 19:55:30.090521 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.090521 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 19:55:30.091937 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.091913 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 19:55:30.092021 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.091965 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 19:55:30.092071 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.092029 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 19:55:30.094685 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.094672 2577 server.go:1268] "Started kubelet" Sep 09 19:55:30.094738 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.094722 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 19:55:30.094836 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.094793 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 19:55:30.094942 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.094892 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 19:55:30.095493 ip-10-0-142-99 systemd[1]: Started Kubernetes Kubelet. Sep 09 19:55:30.096245 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.096117 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 19:55:30.096303 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.096157 2577 server.go:310] "Adding debug handlers to kubelet server" Sep 09 19:55:30.100699 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.100675 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 19:55:30.103327 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.102380 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-99.ec2.internal.18d3bf395d3ea730 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-99.ec2.internal,UID:ip-10-0-142-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-99.ec2.internal,},FirstTimestamp:2026-09-09 19:55:30.094647088 +0000 UTC m=+0.358983792,LastTimestamp:2026-09-09 19:55:30.094647088 +0000 UTC m=+0.358983792,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-99.ec2.internal,}" Sep 09 19:55:30.103812 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.103796 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 19:55:30.104383 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.104364 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 19:55:30.104519 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.104501 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.104989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.104969 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 19:55:30.104989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.104971 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 19:55:30.105107 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105001 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 19:55:30.105107 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105081 2577 reconstruct.go:97] "Volume reconstruction finished" Sep 09 19:55:30.105107 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105087 2577 reconciler.go:29] "Reconciler: start to sync state" Sep 09 19:55:30.105340 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105323 2577 factory.go:55] Registering systemd factory Sep 09 19:55:30.105340 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105339 2577 factory.go:223] Registration of the systemd container factory successfully Sep 09 19:55:30.105533 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105521 2577 factory.go:153] Registering CRI-O factory Sep 09 19:55:30.105533 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105535 2577 factory.go:223] Registration of the crio container factory successfully Sep 09 19:55:30.105670 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105574 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 19:55:30.105670 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105587 2577 factory.go:103] Registering Raw factory Sep 09 19:55:30.105670 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.105599 2577 manager.go:1196] Started watching for new ooms in manager Sep 09 19:55:30.106040 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.106027 2577 manager.go:349] Starting recovery of all containers Sep 09 19:55:30.107463 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.107438 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kdl6" Sep 09 19:55:30.115430 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.115255 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 19:55:30.115430 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.115326 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kdl6" Sep 09 19:55:30.115730 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.115702 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 19:55:30.116549 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.116532 2577 manager.go:354] Recovery completed Sep 09 19:55:30.120360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.120348 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 19:55:30.122310 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.122294 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory" Sep 09 19:55:30.122369 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.122320 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 19:55:30.122369 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.122330 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID" Sep 09 19:55:30.123335 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.123322 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 19:55:30.123335 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.123332 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 19:55:30.123457 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.123346 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 09 19:55:30.123457 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.123345 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 19:55:30.124001 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.123986 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory" Sep 09 19:55:30.124058 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.124013 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 19:55:30.124058 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.124024 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID" Sep 09 19:55:30.125241 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.125229 2577 policy_none.go:49] "None policy: Start" Sep 09 19:55:30.125275 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.125245 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 19:55:30.125275 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.125255 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 19:55:30.127222 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.127211 2577 policy_none.go:47] "Start" Sep 09 19:55:30.165142 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.165126 2577 manager.go:337] "Starting Device Plugin manager" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.165162 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.165171 2577 server.go:87] "Starting device plugin registration server" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.165469 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.165479 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.165651 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.165731 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.165741 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.166205 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 19:55:30.183560 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.166247 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.233542 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.233517 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 19:55:30.234788 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.234762 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 19:55:30.234876 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.234818 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 19:55:30.234876 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.234846 2577 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 19:55:30.234971 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.234891 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 19:55:30.236895 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.236874 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 19:55:30.266070 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.266051 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 19:55:30.266832 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.266817 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory" Sep 09 19:55:30.266909 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.266846 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 19:55:30.266909 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.266857 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID" Sep 09 19:55:30.266909 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.266877 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.275191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.275129 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.275191 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.275150 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-99.ec2.internal\": node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.289486 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.289460 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.335944 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.335920 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"] Sep 09 19:55:30.336030 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.335988 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 19:55:30.336850 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.336836 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory" Sep 09 19:55:30.336932 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.336869 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 19:55:30.336932 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.336884 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID" Sep 09 19:55:30.338932 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.338917 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 19:55:30.339062 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339048 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.339104 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339076 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 19:55:30.339633 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339601 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory" Sep 09 19:55:30.339714 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339644 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 19:55:30.339714 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339658 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID" Sep 09 19:55:30.339714 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339678 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory" Sep 09 19:55:30.339714 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339701 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 19:55:30.339714 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.339716 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID" Sep 09 19:55:30.341901 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.341882 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.341989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.341911 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 19:55:30.342503 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.342489 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory" Sep 09 19:55:30.342564 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.342517 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 19:55:30.342564 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.342539 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID" Sep 09 19:55:30.370258 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.370234 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-99.ec2.internal\" not found" node="ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.374412 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.374397 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-99.ec2.internal\" not found" node="ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.390070 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.390047 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.406917 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.406897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8fd42523dcb4ea1404f0c2f6addc573-config\") pod \"kube-apiserver-proxy-ip-10-0-142-99.ec2.internal\" (UID: \"c8fd42523dcb4ea1404f0c2f6addc573\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.406989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.406925 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dd9a537e322e2b0ac4afe839e3db9ef0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"dd9a537e322e2b0ac4afe839e3db9ef0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.406989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.406942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd9a537e322e2b0ac4afe839e3db9ef0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"dd9a537e322e2b0ac4afe839e3db9ef0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.490295 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.490256 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.507603 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.507577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd9a537e322e2b0ac4afe839e3db9ef0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"dd9a537e322e2b0ac4afe839e3db9ef0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.507682 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.507636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8fd42523dcb4ea1404f0c2f6addc573-config\") pod \"kube-apiserver-proxy-ip-10-0-142-99.ec2.internal\" (UID: \"c8fd42523dcb4ea1404f0c2f6addc573\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.507682 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.507662 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dd9a537e322e2b0ac4afe839e3db9ef0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"dd9a537e322e2b0ac4afe839e3db9ef0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.507682 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.507583 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd9a537e322e2b0ac4afe839e3db9ef0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"dd9a537e322e2b0ac4afe839e3db9ef0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.507773 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.507685 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dd9a537e322e2b0ac4afe839e3db9ef0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"dd9a537e322e2b0ac4afe839e3db9ef0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.507773 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.507714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8fd42523dcb4ea1404f0c2f6addc573-config\") pod \"kube-apiserver-proxy-ip-10-0-142-99.ec2.internal\" (UID: \"c8fd42523dcb4ea1404f0c2f6addc573\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.590983 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.590907 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.673449 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.673414 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.676678 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:30.676657 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:30.691996 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.691975 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.792549 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.792502 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.893140 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.893077 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:30.993583 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:30.993556 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:31.045071 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.045052 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 19:55:31.045434 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.045175 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 19:55:31.069334 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.069316 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 19:55:31.094034 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:31.094016 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:31.119021 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.118667 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 19:50:31 +0000 UTC" deadline="2028-05-01 00:55:30.556656037 +0000 UTC" Sep 09 19:55:31.119021 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.118700 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14380h59m59.43796019s" Sep 09 19:55:31.195048 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:31.194978 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:31.254141 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:31.254106 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8fd42523dcb4ea1404f0c2f6addc573.slice/crio-4ebbbffbfa69c894d694bd624cf6dbd1e529a8cfee3b350339fb766693fcae73 WatchSource:0}: Error finding container 4ebbbffbfa69c894d694bd624cf6dbd1e529a8cfee3b350339fb766693fcae73: Status 404 returned error can't find the container with id 4ebbbffbfa69c894d694bd624cf6dbd1e529a8cfee3b350339fb766693fcae73 Sep 09 19:55:31.258098 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.258082 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 19:55:31.295832 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:31.295798 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found" Sep 09 19:55:31.310297 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.310277 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 19:55:31.361262 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.361239 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 19:55:31.404988 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.404969 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" Sep 09 19:55:31.418524 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.418510 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 19:55:31.419464 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.419451 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" Sep 09 19:55:31.428999 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:31.428981 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9a537e322e2b0ac4afe839e3db9ef0.slice/crio-59cd32f591e9380da70385d0192d69dab45e4de9d813f8a6308cd5543ed6e202 WatchSource:0}: Error finding container 59cd32f591e9380da70385d0192d69dab45e4de9d813f8a6308cd5543ed6e202: Status 404 returned error can't find the container with id 59cd32f591e9380da70385d0192d69dab45e4de9d813f8a6308cd5543ed6e202 Sep 09 19:55:31.432597 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:31.432583 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 19:55:33.687718 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.687497 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 19:55:33.724861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.724832 2577 apiserver.go:52] "Watching apiserver" Sep 09 19:55:33.734663 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.734489 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 19:55:33.735464 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.735418 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal","openshift-multus/multus-additional-cni-plugins-g54lw","openshift-network-diagnostics/network-check-target-wdjfl","openshift-ovn-kubernetes/ovnkube-node-5rrgv","kube-system/global-pull-secret-syncer-r5fjb","kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787","openshift-image-registry/node-ca-mcc67","openshift-multus/multus-grl76","openshift-multus/network-metrics-daemon-98kdp","openshift-network-operator/iptables-alerter-q7w5b","kube-system/konnectivity-agent-wd8dj","openshift-cluster-node-tuning-operator/tuned-b2f4q","openshift-dns/node-resolver-rkj8g"] Sep 09 19:55:33.738031 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.738010 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-grl76" Sep 09 19:55:33.740898 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.740632 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.742762 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.742131 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 19:55:33.742762 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.742304 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 19:55:33.742762 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.742595 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2jq8f\"" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.743742 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l8k4n\"" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.743923 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.744076 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.744352 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.744488 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.744879 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.745339 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.745665 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:33.746861 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.745962 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:33.749353 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.749030 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.751432 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.751413 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.752081 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.752065 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 19:55:33.754212 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.753824 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.754937 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.754919 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 19:55:33.755140 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755121 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 19:55:33.755258 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755121 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 19:55:33.755370 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755349 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 19:55:33.755830 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755809 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-cni-binary-copy\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.755924 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755843 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-socket-dir-parent\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.755924 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-netns\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.755924 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755898 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-hosts-file\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.755924 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-hostroot\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-cni-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755970 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-k8s-cni-cncf-io\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.755991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-cni-bin\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-kubelet\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-etc-kubernetes\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756087 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-cnibin\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-conf-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756132 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-daemon-config\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzfkq\" (UniqueName: \"kubernetes.io/projected/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-kube-api-access-zzfkq\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-tmp-dir\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756208 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756229 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-system-cni-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756247 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-cni-multus\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-multus-certs\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756309 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rdf9\" (UniqueName: \"kubernetes.io/projected/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-kube-api-access-4rdf9\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756345 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk8ps\" (UniqueName: \"kubernetes.io/projected/379c5732-e859-4f78-895c-62bd2f25915b-kube-api-access-qk8ps\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:33.756556 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.756520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.757005 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.756617 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:33.758477 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.758168 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 19:55:33.758683 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.758667 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 19:55:33.758774 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.758740 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 19:55:33.758868 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.758855 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 19:55:33.758991 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.758973 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4lr57\"" Sep 09 19:55:33.759150 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.759134 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.759150 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.759144 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p59rt\"" Sep 09 19:55:33.759234 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.759181 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 19:55:33.759373 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.759357 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 19:55:33.759740 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.759718 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pc922\"" Sep 09 19:55:33.759825 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.759754 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 19:55:33.759825 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.759796 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 19:55:33.761795 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.761770 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:33.764588 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.764571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.766003 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.765949 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 19:55:33.767023 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.767007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.768288 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.768253 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 19:55:33.768398 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.768343 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 19:55:33.768636 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.768614 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k4v5g\"" Sep 09 19:55:33.769390 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769371 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bh682\"" Sep 09 19:55:33.769567 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769550 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wlkmr\"" Sep 09 19:55:33.769693 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769674 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 19:55:33.769779 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769694 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6dhsg\"" Sep 09 19:55:33.769846 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769793 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 19:55:33.770031 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769900 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 19:55:33.770031 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769905 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 19:55:33.770031 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.769968 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 19:55:33.770230 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.770211 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 19:55:33.776533 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.776511 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 19:55:33.804125 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.804104 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tcfqb" Sep 09 19:55:33.815613 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.815594 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tcfqb" Sep 09 19:55:33.846355 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.846319 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 19:55:33.856807 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-cnibin\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.856807 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856809 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-conf-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-run\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-tuned\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7tpf\" (UniqueName: \"kubernetes.io/projected/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-kube-api-access-b7tpf\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78ec08ab-1e3b-4efc-98ed-e659284b71b6-serviceca\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856925 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-etc-selinux\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-cni-multus\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysconfig\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.856989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.856988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-env-overrides\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-socket-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-device-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-hosts-file\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-systemd-units\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-var-lib-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-ovnkube-config\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857182 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6klsj\" (UniqueName: \"kubernetes.io/projected/b8d6c369-e973-463c-9a1d-82626c19b4cb-kube-api-access-6klsj\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857242 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-slash\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857286 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-ovnkube-script-lib\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857309 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkjvv\" (UniqueName: \"kubernetes.io/projected/ea161fd7-651e-4c61-a005-37a870b8e290-kube-api-access-zkjvv\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857324 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-run-netns\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.857368 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk8ps\" (UniqueName: \"kubernetes.io/projected/379c5732-e859-4f78-895c-62bd2f25915b-kube-api-access-qk8ps\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857353 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-var-lib-kubelet\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wfp\" (UniqueName: \"kubernetes.io/projected/3cf5b3a1-2be3-4beb-a6fd-65574667153e-kube-api-access-t6wfp\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857394 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-kubelet\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-socket-dir-parent\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-kubernetes\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857469 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cf5b3a1-2be3-4beb-a6fd-65574667153e-tmp\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-ovn\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-run-ovn-kubernetes\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857544 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-cni-bin\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea161fd7-651e-4c61-a005-37a870b8e290-host-slash\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-k8s-cni-cncf-io\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857641 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-cnibin\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857674 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzfkq\" (UniqueName: \"kubernetes.io/projected/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-kube-api-access-zzfkq\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857743 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-cni-multus\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857745 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-socket-dir-parent\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857746 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-conf-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857767 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-modprobe-d\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857821 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysctl-d\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857826 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-k8s-cni-cncf-io\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857828 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-hosts-file\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857876 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-lib-modules\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857910 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cnibin\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857936 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.857972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d82d5454-6654-4848-b291-fbb6ddc8c6ef-konnectivity-ca\") pod \"konnectivity-agent-wd8dj\" (UID: \"d82d5454-6654-4848-b291-fbb6ddc8c6ef\") " pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-daemon-config\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-log-socket\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea161fd7-651e-4c61-a005-37a870b8e290-iptables-alerter-script\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858149 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-multus-certs\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858242 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-cni-netd\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858277 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-multus-certs\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858313 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/139f0b1d-e70f-458b-8a23-76e024dc6685-ovn-node-metrics-cert\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858346 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78ec08ab-1e3b-4efc-98ed-e659284b71b6-host\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.858785 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-registration-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-netns\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-daemon-config\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858475 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-run-netns\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-sys\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-sys-fs\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858544 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c45f59b-2945-416b-86d6-570d211d5f28-dbus\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-hostroot\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-hostroot\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-cni-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-cni-bin\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858707 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-multus-cni-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858732 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-etc-kubernetes\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-tmp-dir\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858759 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-cni-bin\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-host\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-etc-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.859545 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858835 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-etc-kubernetes\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-node-log\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858906 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl4f5\" (UniqueName: \"kubernetes.io/projected/139f0b1d-e70f-458b-8a23-76e024dc6685-kube-api-access-cl4f5\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d82d5454-6654-4848-b291-fbb6ddc8c6ef-agent-certs\") pod \"konnectivity-agent-wd8dj\" (UID: \"d82d5454-6654-4848-b291-fbb6ddc8c6ef\") " pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c45f59b-2945-416b-86d6-570d211d5f28-kubelet-config\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.858979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-system-cni-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859022 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-tmp-dir\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rdf9\" (UniqueName: \"kubernetes.io/projected/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-kube-api-access-4rdf9\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859061 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysctl-conf\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859087 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-system-cni-dir\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859112 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-cni-binary-copy\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859134 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-systemd\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859158 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-system-cni-dir\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859319 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9z7x\" (UniqueName: \"kubernetes.io/projected/78ec08ab-1e3b-4efc-98ed-e659284b71b6-kube-api-access-w9z7x\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-kubelet\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.860191 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859418 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-host-var-lib-kubelet\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.860857 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:33.860857 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.859539 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:33.860857 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-systemd\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.860857 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.859657 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:55:34.35964263 +0000 UTC m=+2.984146644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:33.860857 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.859933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-cni-binary-copy\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.864118 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.864098 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 19:55:33.864252 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.864127 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 19:55:33.864252 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.864139 2577 projected.go:196] Error preparing data for projected volume kube-api-access-vzmv2 for pod openshift-network-diagnostics/network-check-target-wdjfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:33.864252 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.864207 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2 podName:267a61e5-1990-4b89-9026-0355c180c0ec nodeName:}" failed. No retries permitted until 2026-09-09 19:55:34.364192475 +0000 UTC m=+2.988696491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vzmv2" (UniqueName: "kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2") pod "network-check-target-wdjfl" (UID: "267a61e5-1990-4b89-9026-0355c180c0ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:33.866482 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.866439 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 19:55:33.870905 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.870787 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk8ps\" (UniqueName: \"kubernetes.io/projected/379c5732-e859-4f78-895c-62bd2f25915b-kube-api-access-qk8ps\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:33.871020 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.870904 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzfkq\" (UniqueName: \"kubernetes.io/projected/d6a74307-af6c-4b04-ab8b-ef97b1b0104c-kube-api-access-zzfkq\") pod \"multus-grl76\" (UID: \"d6a74307-af6c-4b04-ab8b-ef97b1b0104c\") " pod="openshift-multus/multus-grl76" Sep 09 19:55:33.871020 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.870819 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rdf9\" (UniqueName: \"kubernetes.io/projected/c34f0e58-b333-4941-b0fb-8d0ec6c461cd-kube-api-access-4rdf9\") pod \"node-resolver-rkj8g\" (UID: \"c34f0e58-b333-4941-b0fb-8d0ec6c461cd\") " pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:33.879819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.879784 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" event={"ID":"dd9a537e322e2b0ac4afe839e3db9ef0","Type":"ContainerStarted","Data":"59cd32f591e9380da70385d0192d69dab45e4de9d813f8a6308cd5543ed6e202"} Sep 09 19:55:33.881180 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.881154 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" event={"ID":"c8fd42523dcb4ea1404f0c2f6addc573","Type":"ContainerStarted","Data":"4ebbbffbfa69c894d694bd624cf6dbd1e529a8cfee3b350339fb766693fcae73"} Sep 09 19:55:33.960524 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-systemd\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.960524 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960498 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-run\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.960524 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960515 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-tuned\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.960741 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7tpf\" (UniqueName: \"kubernetes.io/projected/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-kube-api-access-b7tpf\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.960741 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960535 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-systemd\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.960741 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78ec08ab-1e3b-4efc-98ed-e659284b71b6-serviceca\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.960741 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960659 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-run\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.960741 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-etc-selinux\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.960741 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysconfig\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960768 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-etc-selinux\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-env-overrides\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-socket-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960808 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysconfig\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-device-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-systemd-units\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-var-lib-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-ovnkube-config\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-socket-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klsj\" (UniqueName: \"kubernetes.io/projected/b8d6c369-e973-463c-9a1d-82626c19b4cb-kube-api-access-6klsj\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960949 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-systemd-units\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.960999 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78ec08ab-1e3b-4efc-98ed-e659284b71b6-serviceca\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-device-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.961014 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-slash\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961058 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-var-lib-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-ovnkube-script-lib\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.961099 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:33.961153 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret podName:6c45f59b-2945-416b-86d6-570d211d5f28 nodeName:}" failed. No retries permitted until 2026-09-09 19:55:34.46113603 +0000 UTC m=+3.085640065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret") pod "global-pull-secret-syncer-r5fjb" (UID: "6c45f59b-2945-416b-86d6-570d211d5f28") : object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961161 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-slash\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkjvv\" (UniqueName: \"kubernetes.io/projected/ea161fd7-651e-4c61-a005-37a870b8e290-kube-api-access-zkjvv\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-run-netns\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-var-lib-kubelet\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961311 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wfp\" (UniqueName: \"kubernetes.io/projected/3cf5b3a1-2be3-4beb-a6fd-65574667153e-kube-api-access-t6wfp\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-kubelet\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961371 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-run-netns\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-kubernetes\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cf5b3a1-2be3-4beb-a6fd-65574667153e-tmp\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-ovn\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.961819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-run-ovn-kubernetes\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961511 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-ovnkube-config\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-cni-bin\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961543 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-cni-bin\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961549 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea161fd7-651e-4c61-a005-37a870b8e290-host-slash\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961572 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-modprobe-d\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysctl-d\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-lib-modules\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961613 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cnibin\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961639 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d82d5454-6654-4848-b291-fbb6ddc8c6ef-konnectivity-ca\") pod \"konnectivity-agent-wd8dj\" (UID: \"d82d5454-6654-4848-b291-fbb6ddc8c6ef\") " pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-var-lib-kubelet\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-log-socket\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961679 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-env-overrides\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961687 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea161fd7-651e-4c61-a005-37a870b8e290-iptables-alerter-script\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.962632 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961705 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/139f0b1d-e70f-458b-8a23-76e024dc6685-ovnkube-script-lib\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-cni-netd\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961724 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-kubelet\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-kubernetes\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961755 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-cni-netd\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961757 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-ovn\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961774 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea161fd7-651e-4c61-a005-37a870b8e290-host-slash\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961785 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/139f0b1d-e70f-458b-8a23-76e024dc6685-ovn-node-metrics-cert\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961812 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78ec08ab-1e3b-4efc-98ed-e659284b71b6-host\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-registration-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-sys\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961862 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-modprobe-d\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961899 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-sys-fs\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c45f59b-2945-416b-86d6-570d211d5f28-dbus\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961944 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-host-run-ovn-kubernetes\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961958 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-host\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.961963 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysctl-d\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.963475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-sys-fs\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962023 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-log-socket\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962053 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-sys\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962064 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-host\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962080 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-registration-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-etc-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962106 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c45f59b-2945-416b-86d6-570d211d5f28-dbus\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962117 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962121 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78ec08ab-1e3b-4efc-98ed-e659284b71b6-host\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-node-log\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl4f5\" (UniqueName: \"kubernetes.io/projected/139f0b1d-e70f-458b-8a23-76e024dc6685-kube-api-access-cl4f5\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d82d5454-6654-4848-b291-fbb6ddc8c6ef-agent-certs\") pod \"konnectivity-agent-wd8dj\" (UID: \"d82d5454-6654-4848-b291-fbb6ddc8c6ef\") " pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962173 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c45f59b-2945-416b-86d6-570d211d5f28-kubelet-config\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962334 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-etc-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962370 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-run-openvswitch\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/139f0b1d-e70f-458b-8a23-76e024dc6685-node-log\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c45f59b-2945-416b-86d6-570d211d5f28-kubelet-config\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:33.964278 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysctl-conf\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962562 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-system-cni-dir\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-systemd\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962644 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9z7x\" (UniqueName: \"kubernetes.io/projected/78ec08ab-1e3b-4efc-98ed-e659284b71b6-kube-api-access-w9z7x\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962664 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea161fd7-651e-4c61-a005-37a870b8e290-iptables-alerter-script\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962732 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-system-cni-dir\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962745 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-systemd\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962877 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d82d5454-6654-4848-b291-fbb6ddc8c6ef-konnectivity-ca\") pod \"konnectivity-agent-wd8dj\" (UID: \"d82d5454-6654-4848-b291-fbb6ddc8c6ef\") " pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.962951 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8d6c369-e973-463c-9a1d-82626c19b4cb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.963021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-sysctl-conf\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.963037 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cf5b3a1-2be3-4beb-a6fd-65574667153e-lib-modules\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.963091 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cnibin\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.963191 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.963196 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.964105 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cf5b3a1-2be3-4beb-a6fd-65574667153e-etc-tuned\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.964325 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cf5b3a1-2be3-4beb-a6fd-65574667153e-tmp\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.965171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.964413 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/139f0b1d-e70f-458b-8a23-76e024dc6685-ovn-node-metrics-cert\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.965978 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.964891 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d82d5454-6654-4848-b291-fbb6ddc8c6ef-agent-certs\") pod \"konnectivity-agent-wd8dj\" (UID: \"d82d5454-6654-4848-b291-fbb6ddc8c6ef\") " pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:33.973297 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.973257 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7tpf\" (UniqueName: \"kubernetes.io/projected/97a8e257-3a98-4fb8-b487-d2cc2a1be7bc-kube-api-access-b7tpf\") pod \"multus-additional-cni-plugins-g54lw\" (UID: \"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc\") " pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:33.974770 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.974750 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkjvv\" (UniqueName: \"kubernetes.io/projected/ea161fd7-651e-4c61-a005-37a870b8e290-kube-api-access-zkjvv\") pod \"iptables-alerter-q7w5b\" (UID: \"ea161fd7-651e-4c61-a005-37a870b8e290\") " pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:33.975856 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.975833 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl4f5\" (UniqueName: \"kubernetes.io/projected/139f0b1d-e70f-458b-8a23-76e024dc6685-kube-api-access-cl4f5\") pod \"ovnkube-node-5rrgv\" (UID: \"139f0b1d-e70f-458b-8a23-76e024dc6685\") " pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:33.976563 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.976540 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9z7x\" (UniqueName: \"kubernetes.io/projected/78ec08ab-1e3b-4efc-98ed-e659284b71b6-kube-api-access-w9z7x\") pod \"node-ca-mcc67\" (UID: \"78ec08ab-1e3b-4efc-98ed-e659284b71b6\") " pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:33.976817 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.976797 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wfp\" (UniqueName: \"kubernetes.io/projected/3cf5b3a1-2be3-4beb-a6fd-65574667153e-kube-api-access-t6wfp\") pod \"tuned-b2f4q\" (UID: \"3cf5b3a1-2be3-4beb-a6fd-65574667153e\") " pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:33.977252 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:33.977235 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klsj\" (UniqueName: \"kubernetes.io/projected/b8d6c369-e973-463c-9a1d-82626c19b4cb-kube-api-access-6klsj\") pod \"aws-ebs-csi-driver-node-dw787\" (UID: \"b8d6c369-e973-463c-9a1d-82626c19b4cb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:34.050250 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.050224 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-grl76" Sep 09 19:55:34.057606 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.057589 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkj8g" Sep 09 19:55:34.067379 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.067358 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:34.074960 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.074945 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" Sep 09 19:55:34.082347 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.082331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mcc67" Sep 09 19:55:34.090042 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.090023 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7w5b" Sep 09 19:55:34.097915 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.097897 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:34.104465 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.104431 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" Sep 09 19:55:34.110579 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.110562 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g54lw" Sep 09 19:55:34.365414 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.365332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:34.365593 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.365502 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:34.365593 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.365571 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:55:35.365551357 +0000 UTC m=+3.990055390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:34.366060 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.366031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:34.366251 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.366237 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 19:55:34.366311 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.366258 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 19:55:34.366311 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.366272 2577 projected.go:196] Error preparing data for projected volume kube-api-access-vzmv2 for pod openshift-network-diagnostics/network-check-target-wdjfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:34.366408 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.366350 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2 podName:267a61e5-1990-4b89-9026-0355c180c0ec nodeName:}" failed. No retries permitted until 2026-09-09 19:55:35.366310046 +0000 UTC m=+3.990814062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vzmv2" (UniqueName: "kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2") pod "network-check-target-wdjfl" (UID: "267a61e5-1990-4b89-9026-0355c180c0ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:34.466488 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.466429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:34.466643 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.466571 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:34.466643 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.466640 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret podName:6c45f59b-2945-416b-86d6-570d211d5f28 nodeName:}" failed. No retries permitted until 2026-09-09 19:55:35.466625259 +0000 UTC m=+4.091129285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret") pod "global-pull-secret-syncer-r5fjb" (UID: "6c45f59b-2945-416b-86d6-570d211d5f28") : object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:34.765348 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:34.765324 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82d5454_6654_4848_b291_fbb6ddc8c6ef.slice/crio-e109af468eeae7c50f14bed24d34d48f3247151835b6a72ddfb43a73d601bbbf WatchSource:0}: Error finding container e109af468eeae7c50f14bed24d34d48f3247151835b6a72ddfb43a73d601bbbf: Status 404 returned error can't find the container with id e109af468eeae7c50f14bed24d34d48f3247151835b6a72ddfb43a73d601bbbf Sep 09 19:55:34.773303 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:34.773203 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d6c369_e973_463c_9a1d_82626c19b4cb.slice/crio-8bacb1c1c7dfac2ff62da35768e3944d5246a800b44597cb174d640cf3b96522 WatchSource:0}: Error finding container 8bacb1c1c7dfac2ff62da35768e3944d5246a800b44597cb174d640cf3b96522: Status 404 returned error can't find the container with id 8bacb1c1c7dfac2ff62da35768e3944d5246a800b44597cb174d640cf3b96522 Sep 09 19:55:34.774938 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:34.774915 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a74307_af6c_4b04_ab8b_ef97b1b0104c.slice/crio-2e31ed35b284fc6b942ddeabea6ae8d7bfa5272c7b9648d5888e944d886c6dc7 WatchSource:0}: Error finding container 2e31ed35b284fc6b942ddeabea6ae8d7bfa5272c7b9648d5888e944d886c6dc7: Status 404 returned error can't find the container with id 2e31ed35b284fc6b942ddeabea6ae8d7bfa5272c7b9648d5888e944d886c6dc7 Sep 09 19:55:34.776108 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:55:34.775830 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc34f0e58_b333_4941_b0fb_8d0ec6c461cd.slice/crio-e8fb625eb53f33fe03414875e5cf3b9e315018e8c7062f3fa0019d5d93dc37b3 WatchSource:0}: Error finding container e8fb625eb53f33fe03414875e5cf3b9e315018e8c7062f3fa0019d5d93dc37b3: Status 404 returned error can't find the container with id e8fb625eb53f33fe03414875e5cf3b9e315018e8c7062f3fa0019d5d93dc37b3 Sep 09 19:55:34.819387 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.819346 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 19:50:33 +0000 UTC" deadline="2028-06-11 08:48:45.124608798 +0000 UTC" Sep 09 19:55:34.819387 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.819384 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15372h53m10.305226979s" Sep 09 19:55:34.875493 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.875470 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:34.875664 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:34.875582 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:34.884032 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.883997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-grl76" event={"ID":"d6a74307-af6c-4b04-ab8b-ef97b1b0104c","Type":"ContainerStarted","Data":"2e31ed35b284fc6b942ddeabea6ae8d7bfa5272c7b9648d5888e944d886c6dc7"} Sep 09 19:55:34.885240 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.885206 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkj8g" event={"ID":"c34f0e58-b333-4941-b0fb-8d0ec6c461cd","Type":"ContainerStarted","Data":"e8fb625eb53f33fe03414875e5cf3b9e315018e8c7062f3fa0019d5d93dc37b3"} Sep 09 19:55:34.886506 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.886478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" event={"ID":"b8d6c369-e973-463c-9a1d-82626c19b4cb","Type":"ContainerStarted","Data":"8bacb1c1c7dfac2ff62da35768e3944d5246a800b44597cb174d640cf3b96522"} Sep 09 19:55:34.887606 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.887581 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7w5b" event={"ID":"ea161fd7-651e-4c61-a005-37a870b8e290","Type":"ContainerStarted","Data":"2a327bd9022a3124ae3faf20c623cae113cc9c678fdea19d7c580ea6ea6a7738"} Sep 09 19:55:34.888737 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.888716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mcc67" event={"ID":"78ec08ab-1e3b-4efc-98ed-e659284b71b6","Type":"ContainerStarted","Data":"8b01d8520e223f44227b8ec613b2f197ea6b5869088b8020d9cd48d13012729f"} Sep 09 19:55:34.890942 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.890913 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" event={"ID":"c8fd42523dcb4ea1404f0c2f6addc573","Type":"ContainerStarted","Data":"8557b089565ef7723be5a2f8e04e3a4384116d219ca9634e528ac7b7824a3ed7"} Sep 09 19:55:34.892229 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.892210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"63a73633832f468706c891ec85711c62156236b8cd25723b4bb909e2bc65bc6a"} Sep 09 19:55:34.893502 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.893481 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerStarted","Data":"dc9723402182e6446b9ddb5d8cfa409263ad8a191b89a2f54fda90baaadf0494"} Sep 09 19:55:34.894580 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.894553 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" event={"ID":"3cf5b3a1-2be3-4beb-a6fd-65574667153e","Type":"ContainerStarted","Data":"9f152d59647b22279ad7aacb26d3cb2ed547801cead81dc78b2c9260cc36685d"} Sep 09 19:55:34.895734 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:34.895711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wd8dj" event={"ID":"d82d5454-6654-4848-b291-fbb6ddc8c6ef","Type":"ContainerStarted","Data":"e109af468eeae7c50f14bed24d34d48f3247151835b6a72ddfb43a73d601bbbf"} Sep 09 19:55:35.372133 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.372042 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:35.372133 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.372120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:35.372344 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.372276 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 19:55:35.372344 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.372296 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 19:55:35.372344 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.372311 2577 projected.go:196] Error preparing data for projected volume kube-api-access-vzmv2 for pod openshift-network-diagnostics/network-check-target-wdjfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:35.372526 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.372370 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2 podName:267a61e5-1990-4b89-9026-0355c180c0ec nodeName:}" failed. No retries permitted until 2026-09-09 19:55:37.372352167 +0000 UTC m=+5.996856188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vzmv2" (UniqueName: "kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2") pod "network-check-target-wdjfl" (UID: "267a61e5-1990-4b89-9026-0355c180c0ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:35.372902 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.372882 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:35.372991 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.372940 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:55:37.372925133 +0000 UTC m=+5.997429153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:35.473428 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.473384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:35.473618 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.473603 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:35.473685 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.473669 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret podName:6c45f59b-2945-416b-86d6-570d211d5f28 nodeName:}" failed. No retries permitted until 2026-09-09 19:55:37.473651538 +0000 UTC m=+6.098155570 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret") pod "global-pull-secret-syncer-r5fjb" (UID: "6c45f59b-2945-416b-86d6-570d211d5f28") : object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:35.819698 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.819624 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 19:50:33 +0000 UTC" deadline="2028-04-11 15:52:56.711272306 +0000 UTC" Sep 09 19:55:35.819698 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.819663 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13915h57m20.891614379s" Sep 09 19:55:35.878211 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.877350 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:35.878211 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.877489 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:35.878211 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.877565 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:35.878211 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:35.877633 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:35.904559 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.904489 2577 generic.go:356] "Generic (PLEG): container finished" podID="dd9a537e322e2b0ac4afe839e3db9ef0" containerID="e1f5ff82df9cb24773db76f1b9ef04339d81bc6ae553f17db6f012269029d547" exitCode=0 Sep 09 19:55:35.905397 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.905369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" event={"ID":"dd9a537e322e2b0ac4afe839e3db9ef0","Type":"ContainerDied","Data":"e1f5ff82df9cb24773db76f1b9ef04339d81bc6ae553f17db6f012269029d547"} Sep 09 19:55:35.923030 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:35.922979 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" podStartSLOduration=2.9229620389999997 podStartE2EDuration="2.922962039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:34.908877629 +0000 UTC m=+3.533381681" watchObservedRunningTime="2026-09-09 19:55:35.922962039 +0000 UTC m=+4.547466080" Sep 09 19:55:36.875801 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:36.875496 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:36.876267 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:36.875932 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:36.915561 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:36.915518 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" event={"ID":"dd9a537e322e2b0ac4afe839e3db9ef0","Type":"ContainerStarted","Data":"807a4e60cbfbadfe9f9b85d517b0d4598eed70b4af6d8cbea55e0b70d02dd0ef"} Sep 09 19:55:36.931687 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:36.931636 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" podStartSLOduration=3.93161842 podStartE2EDuration="3.93161842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:36.931354772 +0000 UTC m=+5.555858810" watchObservedRunningTime="2026-09-09 19:55:36.93161842 +0000 UTC m=+5.556122457" Sep 09 19:55:37.394233 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:37.394192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:37.394426 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:37.394270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:37.394604 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.394586 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:37.394757 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.394743 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:55:41.394721977 +0000 UTC m=+10.019225994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:37.395154 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.394442 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 19:55:37.395255 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.395164 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 19:55:37.395255 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.395180 2577 projected.go:196] Error preparing data for projected volume kube-api-access-vzmv2 for pod openshift-network-diagnostics/network-check-target-wdjfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:37.395255 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.395237 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2 podName:267a61e5-1990-4b89-9026-0355c180c0ec nodeName:}" failed. No retries permitted until 2026-09-09 19:55:41.395222314 +0000 UTC m=+10.019726334 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vzmv2" (UniqueName: "kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2") pod "network-check-target-wdjfl" (UID: "267a61e5-1990-4b89-9026-0355c180c0ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:37.494927 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:37.494888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:37.495067 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.495058 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:37.495115 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.495102 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret podName:6c45f59b-2945-416b-86d6-570d211d5f28 nodeName:}" failed. No retries permitted until 2026-09-09 19:55:41.495091124 +0000 UTC m=+10.119595139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret") pod "global-pull-secret-syncer-r5fjb" (UID: "6c45f59b-2945-416b-86d6-570d211d5f28") : object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:37.875893 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:37.875817 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:37.876258 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.875936 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:37.876323 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:37.876306 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:37.876420 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:37.876401 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:38.875633 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:38.875107 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:38.875633 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:38.875263 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:39.875286 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:39.875251 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:39.875777 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:39.875387 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:39.877891 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:39.875222 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:39.877891 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:39.877840 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:40.875643 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:40.875560 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:40.876067 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:40.875745 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:41.426682 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:41.426644 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:41.426867 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:41.426729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:41.426867 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.426853 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:41.426971 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.426913 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:55:49.426895264 +0000 UTC m=+18.051399283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:41.427923 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.427895 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 19:55:41.428050 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.427922 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 19:55:41.428050 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.427944 2577 projected.go:196] Error preparing data for projected volume kube-api-access-vzmv2 for pod openshift-network-diagnostics/network-check-target-wdjfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:41.428050 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.427996 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2 podName:267a61e5-1990-4b89-9026-0355c180c0ec nodeName:}" failed. No retries permitted until 2026-09-09 19:55:49.427972599 +0000 UTC m=+18.052476615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vzmv2" (UniqueName: "kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2") pod "network-check-target-wdjfl" (UID: "267a61e5-1990-4b89-9026-0355c180c0ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:41.527381 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:41.527342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:41.527580 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.527535 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:41.527656 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.527626 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret podName:6c45f59b-2945-416b-86d6-570d211d5f28 nodeName:}" failed. No retries permitted until 2026-09-09 19:55:49.527583819 +0000 UTC m=+18.152087856 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret") pod "global-pull-secret-syncer-r5fjb" (UID: "6c45f59b-2945-416b-86d6-570d211d5f28") : object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:41.876102 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:41.876021 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:41.876555 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:41.876108 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:41.876555 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.876135 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:41.876555 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:41.876230 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:42.875660 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:42.875187 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:42.875660 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:42.875326 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:43.875128 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:43.875093 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:43.875626 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:43.875229 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:43.875626 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:43.875485 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:43.875626 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:43.875586 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:44.875418 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:44.875381 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:44.875875 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:44.875560 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:45.875950 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:45.875899 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:45.876338 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:45.876024 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:45.876399 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:45.876373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:45.876483 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:45.876438 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:46.875833 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:46.875791 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:46.876028 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:46.875934 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:47.877229 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:47.877147 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:47.877626 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:47.877159 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:47.877626 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:47.877274 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:47.877626 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:47.877338 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:48.875844 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:48.875767 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:48.876054 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:48.875897 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:49.488710 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:49.488677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:49.489118 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:49.488743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:49.489118 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.488900 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 19:55:49.489118 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.488916 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 19:55:49.489118 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.488929 2577 projected.go:196] Error preparing data for projected volume kube-api-access-vzmv2 for pod openshift-network-diagnostics/network-check-target-wdjfl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:49.489118 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.488984 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2 podName:267a61e5-1990-4b89-9026-0355c180c0ec nodeName:}" failed. No retries permitted until 2026-09-09 19:56:05.488965882 +0000 UTC m=+34.113469969 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vzmv2" (UniqueName: "kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2") pod "network-check-target-wdjfl" (UID: "267a61e5-1990-4b89-9026-0355c180c0ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 19:55:49.489118 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.489051 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:49.489118 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.489082 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:56:05.489071004 +0000 UTC m=+34.113575028 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 19:55:49.589470 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:49.589426 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:49.589633 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.589573 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:49.589681 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.589646 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret podName:6c45f59b-2945-416b-86d6-570d211d5f28 nodeName:}" failed. No retries permitted until 2026-09-09 19:56:05.589626757 +0000 UTC m=+34.214130795 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret") pod "global-pull-secret-syncer-r5fjb" (UID: "6c45f59b-2945-416b-86d6-570d211d5f28") : object "kube-system"/"original-pull-secret" not registered Sep 09 19:55:49.875537 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:49.875463 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:49.875705 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:49.875466 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:49.875705 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.875614 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:49.875705 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:49.875690 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:50.875079 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:50.875047 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:50.875523 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:50.875175 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:51.876661 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:51.875059 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:51.876661 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:51.875177 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:51.877164 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:51.877130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:51.877255 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:51.877237 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:52.862631 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.862362 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 19:55:52.875199 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.875181 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:52.875316 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:52.875292 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:52.970547 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.970508 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"1fd8d475359142b6b9325fea311731c8c0e07dd3d205e6955426f94b7bb0aea6"} Sep 09 19:55:52.971254 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.970602 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"48b6e618f2d85b7bc98139ddeb0bb9e163aa807262a7b8ba6060a45edf1ba0e2"} Sep 09 19:55:52.971254 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.970656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"4d249e6fcc110cf9abb269c396d13bd46a76b9c6a32a90ec073d9a22d8a11614"} Sep 09 19:55:52.971254 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.970704 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"730fef0db5a05005e72433d1b76390f3554f70cdfaaaead46e2963a6fb3af5b4"} Sep 09 19:55:52.971254 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.970742 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"869b3da9a07d9a8537a47c9bfbcf9cdcd924c0b3b25b5f67010574c15659a3f4"} Sep 09 19:55:52.971254 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.970774 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"277dfb892a15bcf3ae05a89060df8adf5ca8926441be98d6ce74f5dff218421f"} Sep 09 19:55:52.972045 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.972019 2577 generic.go:356] "Generic (PLEG): container finished" podID="97a8e257-3a98-4fb8-b487-d2cc2a1be7bc" containerID="be522a6343f14f65d270e7a70daea5e4647a34e8b1a5eb25374ff43f7365ac6b" exitCode=0 Sep 09 19:55:52.972147 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.972108 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerDied","Data":"be522a6343f14f65d270e7a70daea5e4647a34e8b1a5eb25374ff43f7365ac6b"} Sep 09 19:55:52.973607 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.973589 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" event={"ID":"3cf5b3a1-2be3-4beb-a6fd-65574667153e","Type":"ContainerStarted","Data":"ce69e954603a78c14d66fdfbd6a2ae3ba44d74cd4dbda1a658c8284eb5126165"} Sep 09 19:55:52.975104 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.975085 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wd8dj" event={"ID":"d82d5454-6654-4848-b291-fbb6ddc8c6ef","Type":"ContainerStarted","Data":"6f33cebdf5296b57afcbb157410d0a7a39941790c7bed7e3530866ed98a67e72"} Sep 09 19:55:52.976608 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.976584 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-grl76" event={"ID":"d6a74307-af6c-4b04-ab8b-ef97b1b0104c","Type":"ContainerStarted","Data":"719acbae84dfd524c6c905ccb0480bc8f4e837bd86a0e3442846941ea32965b8"} Sep 09 19:55:52.978422 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.978397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkj8g" event={"ID":"c34f0e58-b333-4941-b0fb-8d0ec6c461cd","Type":"ContainerStarted","Data":"174c02f0a270c0f83e1c9980746c5ccf72d2b26c6d3234c355ab476b77a246ac"} Sep 09 19:55:52.980065 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.980045 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" event={"ID":"b8d6c369-e973-463c-9a1d-82626c19b4cb","Type":"ContainerStarted","Data":"1f375612d9f12b2ae0605a881f11a52ec28a6f676ad78bcd94cb6a1bf23b7afa"} Sep 09 19:55:52.980157 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.980091 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" event={"ID":"b8d6c369-e973-463c-9a1d-82626c19b4cb","Type":"ContainerStarted","Data":"fba4a56b374bdbdc4e4449847ebf0a3834b693575e3d9bda8f6020627bcaf038"} Sep 09 19:55:52.981271 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:52.981254 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mcc67" event={"ID":"78ec08ab-1e3b-4efc-98ed-e659284b71b6","Type":"ContainerStarted","Data":"39deb2fb9ecde192adea4c826311f92c6f71fba93fd44ee2e3311115844b85c8"} Sep 09 19:55:53.014812 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.014727 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-grl76" podStartSLOduration=4.934707719 podStartE2EDuration="22.014715584s" totalImagesPullingTime="17.080007865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:53.014435078 +0000 UTC m=+21.638939116" watchObservedRunningTime="2026-09-09 19:55:53.014715584 +0000 UTC m=+21.639219620" Sep 09 19:55:53.044282 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.044246 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wd8dj" podStartSLOduration=8.401784929 podStartE2EDuration="21.044233235s" totalImagesPullingTime="12.642448306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:53.029432426 +0000 UTC m=+21.653936457" watchObservedRunningTime="2026-09-09 19:55:53.044233235 +0000 UTC m=+21.668737271" Sep 09 19:55:53.044427 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.044404 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rkj8g" podStartSLOduration=5.014483614 podStartE2EDuration="22.04439601s" totalImagesPullingTime="17.029912396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:53.044138847 +0000 UTC m=+21.668642875" watchObservedRunningTime="2026-09-09 19:55:53.04439601 +0000 UTC m=+21.668900048" Sep 09 19:55:53.061602 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.059250 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mcc67" podStartSLOduration=4.02318819 podStartE2EDuration="21.05919151s" totalImagesPullingTime="17.03600332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:53.058794206 +0000 UTC m=+21.683298235" watchObservedRunningTime="2026-09-09 19:55:53.05919151 +0000 UTC m=+21.683695548" Sep 09 19:55:53.081500 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.081439 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b2f4q" podStartSLOduration=4.040602564 podStartE2EDuration="21.081425539s" totalImagesPullingTime="17.040822975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:53.081362279 +0000 UTC m=+21.705866315" watchObservedRunningTime="2026-09-09 19:55:53.081425539 +0000 UTC m=+21.705929575" Sep 09 19:55:53.814163 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.814045 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T19:55:52.862627474Z","UUID":"1ddd5e20-f15c-42e9-b5bc-d44f789dade1","Handler":null,"Name":"","Endpoint":""} Sep 09 19:55:53.816344 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.816322 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 19:55:53.816481 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.816359 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 19:55:53.876026 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.875620 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:53.876026 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.875654 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:53.876026 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:53.875743 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:53.876026 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:53.875865 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:53.986298 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.986220 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" event={"ID":"b8d6c369-e973-463c-9a1d-82626c19b4cb","Type":"ContainerStarted","Data":"1e9bb7ed333a409d1ab482008fae3176fb4cb29b1283376523d4fb7623f8b0ae"} Sep 09 19:55:53.989382 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:53.989353 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7w5b" event={"ID":"ea161fd7-651e-4c61-a005-37a870b8e290","Type":"ContainerStarted","Data":"eb69bc6c1162cca956f0d8be1c39dc9c7215480e5d8c666c7961f6d7ec9d6e45"} Sep 09 19:55:54.006604 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:54.006548 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dw787" podStartSLOduration=3.363082751 podStartE2EDuration="22.006528365s" totalImagesPullingTime="18.643445614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:54.006228797 +0000 UTC m=+22.630732833" watchObservedRunningTime="2026-09-09 19:55:54.006528365 +0000 UTC m=+22.631032403" Sep 09 19:55:54.021739 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:54.021663 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q7w5b" podStartSLOduration=4.981151875 podStartE2EDuration="22.021652016s" totalImagesPullingTime="17.040500141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:54.021094998 +0000 UTC m=+22.645599035" watchObservedRunningTime="2026-09-09 19:55:54.021652016 +0000 UTC m=+22.646156053" Sep 09 19:55:54.875270 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:54.875195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:54.875427 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:54.875329 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:54.995957 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:54.995918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"eed07d97178110cb5b48f6828571f0c4877bd193b379dab2940ad9ee0699e2ed"} Sep 09 19:55:55.875601 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:55.875567 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:55.875813 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:55.875565 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:55.875813 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:55.875695 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:55.875922 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:55.875858 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:56.740690 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:56.740608 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:56.741318 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:56.741290 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:56.874961 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:56.874924 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:56.875113 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:56.875081 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:57.875160 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:57.874970 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:57.875801 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:57.875069 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:57.875801 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:57.875267 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:57.875801 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:57.875296 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:58.006008 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.005971 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" event={"ID":"139f0b1d-e70f-458b-8a23-76e024dc6685","Type":"ContainerStarted","Data":"69472885f4806612d8ff34d5b342942f456a159fb85cf125b1aee466d451e50f"} Sep 09 19:55:58.006216 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.006196 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:58.006282 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.006269 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:58.007945 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.007917 2577 generic.go:356] "Generic (PLEG): container finished" podID="97a8e257-3a98-4fb8-b487-d2cc2a1be7bc" containerID="8b6b36af281ca28e7e4ba3fca08e7b3ab047a0e55f2d7ef3e614265cf7d6c032" exitCode=0 Sep 09 19:55:58.008056 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.007966 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerDied","Data":"8b6b36af281ca28e7e4ba3fca08e7b3ab047a0e55f2d7ef3e614265cf7d6c032"} Sep 09 19:55:58.022713 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.022693 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:58.034079 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.034044 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" podStartSLOduration=8.909729882 podStartE2EDuration="26.03403161s" totalImagesPullingTime="17.124301728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:55:58.032775185 +0000 UTC m=+26.657279224" watchObservedRunningTime="2026-09-09 19:55:58.03403161 +0000 UTC m=+26.658535647" Sep 09 19:55:58.875509 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.875329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:58.875892 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:58.875591 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:58.878652 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.878630 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r5fjb"] Sep 09 19:55:58.878783 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.878722 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:55:58.878833 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:58.878799 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:55:58.882183 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.882160 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wdjfl"] Sep 09 19:55:58.882537 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.882269 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:55:58.882537 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:58.882371 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:55:58.882870 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:58.882823 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98kdp"] Sep 09 19:55:59.012586 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.012503 2577 generic.go:356] "Generic (PLEG): container finished" podID="97a8e257-3a98-4fb8-b487-d2cc2a1be7bc" containerID="369113ee4ea9aa7dd0923e61f276e9c45f97bef201a5a2a3192a3f7d51753417" exitCode=0 Sep 09 19:55:59.012737 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.012587 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerDied","Data":"369113ee4ea9aa7dd0923e61f276e9c45f97bef201a5a2a3192a3f7d51753417"} Sep 09 19:55:59.012737 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.012621 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:55:59.012737 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:55:59.012706 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:55:59.013704 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.013220 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:59.029381 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.029361 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:55:59.210851 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.210825 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:55:59.211075 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.210968 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 19:55:59.211506 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:55:59.211487 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wd8dj" Sep 09 19:56:00.017351 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:00.017264 2577 generic.go:356] "Generic (PLEG): container finished" podID="97a8e257-3a98-4fb8-b487-d2cc2a1be7bc" containerID="36e6c4b565e1e88049c5fd59e76cd2bccbd3d821281caa6235b3569aa6d683c1" exitCode=0 Sep 09 19:56:00.017351 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:00.017329 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerDied","Data":"36e6c4b565e1e88049c5fd59e76cd2bccbd3d821281caa6235b3569aa6d683c1"} Sep 09 19:56:00.875937 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:00.875859 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:00.875937 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:00.875899 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:56:00.876117 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:00.875859 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:56:00.876117 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:00.875965 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:56:00.876117 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:00.876027 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:56:00.876117 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:00.876076 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:56:02.875159 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:02.875119 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:56:02.875159 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:02.875144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:56:02.875159 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:02.875148 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:02.875712 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:02.875263 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:56:02.875712 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:02.875420 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r5fjb" podUID="6c45f59b-2945-416b-86d6-570d211d5f28" Sep 09 19:56:02.875712 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:02.875526 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wdjfl" podUID="267a61e5-1990-4b89-9026-0355c180c0ec" Sep 09 19:56:04.612424 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.612356 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeReady" Sep 09 19:56:04.612848 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.612532 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 19:56:04.663947 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.663912 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ldsrh"] Sep 09 19:56:04.681200 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.680976 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-84qc6"] Sep 09 19:56:04.684484 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.683186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.685989 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.685967 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4z7jm\"" Sep 09 19:56:04.686225 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.686204 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 19:56:04.686578 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.686532 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 19:56:04.693471 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.693435 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ldsrh"] Sep 09 19:56:04.693591 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.693480 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84qc6"] Sep 09 19:56:04.693591 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.693530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:04.696375 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.696339 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 19:56:04.696493 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.696422 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hd98l\"" Sep 09 19:56:04.696563 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.696497 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 19:56:04.696730 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.696716 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 19:56:04.800241 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.800212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-config-volume\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.800241 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.800241 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.800500 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.800276 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-tmp-dir\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.800500 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.800400 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr8h7\" (UniqueName: \"kubernetes.io/projected/0e1e15a7-89e0-4945-9a6b-1e1d97f9648a-kube-api-access-nr8h7\") pod \"ingress-canary-84qc6\" (UID: \"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a\") " pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:04.800500 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.800444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8wkb\" (UniqueName: \"kubernetes.io/projected/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-kube-api-access-m8wkb\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.800630 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.800500 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e1e15a7-89e0-4945-9a6b-1e1d97f9648a-cert\") pod \"ingress-canary-84qc6\" (UID: \"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a\") " pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:04.875803 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.875727 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:56:04.875941 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.875737 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:04.875941 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.875864 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:56:04.878778 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.878757 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bbhmn\"" Sep 09 19:56:04.878778 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.878757 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 19:56:04.879043 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.878807 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 19:56:04.879043 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.878848 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 19:56:04.879043 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.878879 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cpmqx\"" Sep 09 19:56:04.879043 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.878910 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 19:56:04.901705 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.901682 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr8h7\" (UniqueName: \"kubernetes.io/projected/0e1e15a7-89e0-4945-9a6b-1e1d97f9648a-kube-api-access-nr8h7\") pod \"ingress-canary-84qc6\" (UID: \"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a\") " pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:04.901820 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.901718 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8wkb\" (UniqueName: \"kubernetes.io/projected/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-kube-api-access-m8wkb\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.901820 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.901745 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e1e15a7-89e0-4945-9a6b-1e1d97f9648a-cert\") pod \"ingress-canary-84qc6\" (UID: \"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a\") " pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:04.901820 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.901769 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-config-volume\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.901820 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.901791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.902017 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.901835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-tmp-dir\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.902017 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:04.901957 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:56:04.902117 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:04.902023 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:56:05.402002254 +0000 UTC m=+34.026506294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:56:04.902176 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.902148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-tmp-dir\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.902414 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.902389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-config-volume\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.906648 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.906627 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e1e15a7-89e0-4945-9a6b-1e1d97f9648a-cert\") pod \"ingress-canary-84qc6\" (UID: \"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a\") " pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:04.911701 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.911672 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8wkb\" (UniqueName: \"kubernetes.io/projected/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-kube-api-access-m8wkb\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:04.912158 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:04.912133 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr8h7\" (UniqueName: \"kubernetes.io/projected/0e1e15a7-89e0-4945-9a6b-1e1d97f9648a-kube-api-access-nr8h7\") pod \"ingress-canary-84qc6\" (UID: \"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a\") " pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:05.002950 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.002919 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84qc6" Sep 09 19:56:05.328815 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.328599 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84qc6"] Sep 09 19:56:05.333669 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:56:05.333637 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1e15a7_89e0_4945_9a6b_1e1d97f9648a.slice/crio-3bfff6d11773cfd342e60039caf52908c7fd8e2cb57982a541b9d3cd3ad15a90 WatchSource:0}: Error finding container 3bfff6d11773cfd342e60039caf52908c7fd8e2cb57982a541b9d3cd3ad15a90: Status 404 returned error can't find the container with id 3bfff6d11773cfd342e60039caf52908c7fd8e2cb57982a541b9d3cd3ad15a90 Sep 09 19:56:05.409819 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.409788 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:05.409978 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:05.409959 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:56:05.410049 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:05.410037 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:56:06.410016391 +0000 UTC m=+35.034520407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:56:05.510475 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.510417 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:56:05.510673 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.510501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:05.510673 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:05.510579 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 19:56:05.510673 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:05.510653 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:56:37.510631305 +0000 UTC m=+66.135135326 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : secret "metrics-daemon-secret" not found Sep 09 19:56:05.514341 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.514317 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzmv2\" (UniqueName: \"kubernetes.io/projected/267a61e5-1990-4b89-9026-0355c180c0ec-kube-api-access-vzmv2\") pod \"network-check-target-wdjfl\" (UID: \"267a61e5-1990-4b89-9026-0355c180c0ec\") " pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:05.611275 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.611187 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:56:05.613953 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.613927 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c45f59b-2945-416b-86d6-570d211d5f28-original-pull-secret\") pod \"global-pull-secret-syncer-r5fjb\" (UID: \"6c45f59b-2945-416b-86d6-570d211d5f28\") " pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:56:05.791748 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.791712 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:05.796892 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:05.796866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r5fjb" Sep 09 19:56:06.033414 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:06.033363 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84qc6" event={"ID":"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a","Type":"ContainerStarted","Data":"3bfff6d11773cfd342e60039caf52908c7fd8e2cb57982a541b9d3cd3ad15a90"} Sep 09 19:56:06.082274 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:06.082246 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wdjfl"] Sep 09 19:56:06.085584 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:56:06.085558 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267a61e5_1990_4b89_9026_0355c180c0ec.slice/crio-5102b8e5ef95f62b55f9b5abc24fdbc20240d383cb97ea608efe09bef882b160 WatchSource:0}: Error finding container 5102b8e5ef95f62b55f9b5abc24fdbc20240d383cb97ea608efe09bef882b160: Status 404 returned error can't find the container with id 5102b8e5ef95f62b55f9b5abc24fdbc20240d383cb97ea608efe09bef882b160 Sep 09 19:56:06.104979 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:06.104957 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r5fjb"] Sep 09 19:56:06.108529 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:56:06.108502 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c45f59b_2945_416b_86d6_570d211d5f28.slice/crio-1bfb9eaefaa53f587341fa4ce67db1daec56ef3ed8edfc22f765c6a0c3a75ef6 WatchSource:0}: Error finding container 1bfb9eaefaa53f587341fa4ce67db1daec56ef3ed8edfc22f765c6a0c3a75ef6: Status 404 returned error can't find the container with id 1bfb9eaefaa53f587341fa4ce67db1daec56ef3ed8edfc22f765c6a0c3a75ef6 Sep 09 19:56:06.416271 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:06.416191 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:06.416470 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:06.416402 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:56:06.416537 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:06.416483 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:56:08.416446201 +0000 UTC m=+37.040950226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:56:07.037115 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:07.037074 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r5fjb" event={"ID":"6c45f59b-2945-416b-86d6-570d211d5f28","Type":"ContainerStarted","Data":"1bfb9eaefaa53f587341fa4ce67db1daec56ef3ed8edfc22f765c6a0c3a75ef6"} Sep 09 19:56:07.038334 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:07.038313 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wdjfl" event={"ID":"267a61e5-1990-4b89-9026-0355c180c0ec","Type":"ContainerStarted","Data":"5102b8e5ef95f62b55f9b5abc24fdbc20240d383cb97ea608efe09bef882b160"} Sep 09 19:56:08.432967 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:08.432919 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:08.433492 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:08.433179 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:56:08.433492 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:08.433246 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:56:12.433227277 +0000 UTC m=+41.057731292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:56:10.049948 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:10.049688 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84qc6" event={"ID":"0e1e15a7-89e0-4945-9a6b-1e1d97f9648a","Type":"ContainerStarted","Data":"9c13c39afc0ab9e88e0693bff69d1c518391b2a376779e6e5eb86c28b521d0d2"} Sep 09 19:56:10.053610 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:10.053579 2577 generic.go:356] "Generic (PLEG): container finished" podID="97a8e257-3a98-4fb8-b487-d2cc2a1be7bc" containerID="c0088ffa0eb4aaf360865cf07284ad1f59a348dcb0259facdce8504eea4aa238" exitCode=0 Sep 09 19:56:10.053727 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:10.053637 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerDied","Data":"c0088ffa0eb4aaf360865cf07284ad1f59a348dcb0259facdce8504eea4aa238"} Sep 09 19:56:10.068020 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:10.067966 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-84qc6" podStartSLOduration=2.158054918 podStartE2EDuration="6.067951533s" totalImagesPullingTime="3.909896615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:56:10.067073287 +0000 UTC m=+38.691577325" watchObservedRunningTime="2026-09-09 19:56:10.067951533 +0000 UTC m=+38.692455569" Sep 09 19:56:12.465592 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:12.465551 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:12.465968 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:12.465700 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:56:12.465968 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:12.465767 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:56:20.465746542 +0000 UTC m=+49.090250574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:56:13.065305 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:13.065278 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerStarted","Data":"754e979c4e2840d706ef43ba099978ca5015f7f452facfc92c7dbd0af5db88a9"} Sep 09 19:56:13.066914 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:13.066888 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r5fjb" event={"ID":"6c45f59b-2945-416b-86d6-570d211d5f28","Type":"ContainerStarted","Data":"9f2de18285ae198453841500ab989af3389b101aa2eb41de286562f5bedf19db"} Sep 09 19:56:13.068854 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:13.068817 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wdjfl" event={"ID":"267a61e5-1990-4b89-9026-0355c180c0ec","Type":"ContainerStarted","Data":"1977b9e4650ef0bce396525f135eb93d2f989748ddac8ce5bdb84e05e03d78b4"} Sep 09 19:56:13.069285 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:13.069263 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:13.110603 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:13.110528 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wdjfl" podStartSLOduration=35.315886793 podStartE2EDuration="42.110512693s" totalImagesPullingTime="6.7946259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:56:13.109946557 +0000 UTC m=+41.734450593" watchObservedRunningTime="2026-09-09 19:56:13.110512693 +0000 UTC m=+41.735016730" Sep 09 19:56:14.074681 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:14.074649 2577 generic.go:356] "Generic (PLEG): container finished" podID="97a8e257-3a98-4fb8-b487-d2cc2a1be7bc" containerID="754e979c4e2840d706ef43ba099978ca5015f7f452facfc92c7dbd0af5db88a9" exitCode=0 Sep 09 19:56:14.075066 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:14.074735 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerDied","Data":"754e979c4e2840d706ef43ba099978ca5015f7f452facfc92c7dbd0af5db88a9"} Sep 09 19:56:15.080539 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:15.080498 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g54lw" event={"ID":"97a8e257-3a98-4fb8-b487-d2cc2a1be7bc","Type":"ContainerStarted","Data":"e4580d6273412682b9855ae2770fe8ba3a7cfdb41bbe078823b274a27212feab"} Sep 09 19:56:15.102939 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:15.102887 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g54lw" podStartSLOduration=11.573242555 podStartE2EDuration="43.102868854s" totalImagesPullingTime="31.529626299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:56:15.10287065 +0000 UTC m=+43.727374688" watchObservedRunningTime="2026-09-09 19:56:15.102868854 +0000 UTC m=+43.727372891" Sep 09 19:56:15.103415 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:15.103386 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-r5fjb" podStartSLOduration=36.319858708 podStartE2EDuration="43.103378679s" totalImagesPullingTime="6.783519971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:56:14.124014037 +0000 UTC m=+42.748518074" watchObservedRunningTime="2026-09-09 19:56:15.103378679 +0000 UTC m=+43.727882716" Sep 09 19:56:20.517560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:20.517516 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:20.518041 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:20.517662 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:56:20.518041 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:20.517730 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:56:36.517714429 +0000 UTC m=+65.142218444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:56:31.030857 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:31.030830 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5rrgv" Sep 09 19:56:36.520107 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:36.520058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:56:36.520546 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:36.520182 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:56:36.520546 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:36.520245 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:57:08.520231024 +0000 UTC m=+97.144735038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:56:37.526439 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:37.526385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:56:37.526833 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:37.526537 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 19:56:37.526833 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:56:37.526597 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:57:41.526581847 +0000 UTC m=+130.151085862 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : secret "metrics-daemon-secret" not found Sep 09 19:56:40.793578 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.793542 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4"] Sep 09 19:56:40.825584 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.825556 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4"] Sep 09 19:56:40.825715 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.825649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:40.830270 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.830242 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 09 19:56:40.830420 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.830279 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 19:56:40.830420 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.830242 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 19:56:40.831028 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.831007 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 19:56:40.946773 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.946744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhxz8\" (UniqueName: \"kubernetes.io/projected/3dbb1538-5c31-46cd-b736-e32ef434b3bb-kube-api-access-jhxz8\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:40.946941 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.946786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3dbb1538-5c31-46cd-b736-e32ef434b3bb-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:40.946941 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:40.946817 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbb1538-5c31-46cd-b736-e32ef434b3bb-tmp\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.047144 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.047083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhxz8\" (UniqueName: \"kubernetes.io/projected/3dbb1538-5c31-46cd-b736-e32ef434b3bb-kube-api-access-jhxz8\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.047144 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.047125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3dbb1538-5c31-46cd-b736-e32ef434b3bb-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.047263 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.047170 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbb1538-5c31-46cd-b736-e32ef434b3bb-tmp\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.047542 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.047526 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbb1538-5c31-46cd-b736-e32ef434b3bb-tmp\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.050264 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.050243 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3dbb1538-5c31-46cd-b736-e32ef434b3bb-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.056892 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.056870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhxz8\" (UniqueName: \"kubernetes.io/projected/3dbb1538-5c31-46cd-b736-e32ef434b3bb-kube-api-access-jhxz8\") pod \"klusterlet-addon-workmgr-5d59796848-qpkr4\" (UID: \"3dbb1538-5c31-46cd-b736-e32ef434b3bb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.137786 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.137761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:41.428879 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:41.428852 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4"] Sep 09 19:56:41.431564 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:56:41.431536 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbb1538_5c31_46cd_b736_e32ef434b3bb.slice/crio-17baf693f439aad3f76edce4f9e86f7c270faa3fead77f5eaf741be52afae25e WatchSource:0}: Error finding container 17baf693f439aad3f76edce4f9e86f7c270faa3fead77f5eaf741be52afae25e: Status 404 returned error can't find the container with id 17baf693f439aad3f76edce4f9e86f7c270faa3fead77f5eaf741be52afae25e Sep 09 19:56:42.146521 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:42.146481 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" event={"ID":"3dbb1538-5c31-46cd-b736-e32ef434b3bb","Type":"ContainerStarted","Data":"17baf693f439aad3f76edce4f9e86f7c270faa3fead77f5eaf741be52afae25e"} Sep 09 19:56:45.083308 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:45.083279 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wdjfl" Sep 09 19:56:46.158957 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:46.158923 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" event={"ID":"3dbb1538-5c31-46cd-b736-e32ef434b3bb","Type":"ContainerStarted","Data":"38c955c14ab731b628f32fac4474350c93796eedd0361dae12020a2ed58c1a9b"} Sep 09 19:56:46.159351 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:46.159141 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:46.160687 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:46.160670 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:56:46.178921 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:56:46.178880 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" podStartSLOduration=2.252621856 podStartE2EDuration="6.17886824s" totalImagesPullingTime="3.926246384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:56:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:56:46.178650074 +0000 UTC m=+74.803154110" watchObservedRunningTime="2026-09-09 19:56:46.17886824 +0000 UTC m=+74.803372274" Sep 09 19:57:08.524002 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:57:08.523964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:57:08.524384 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:57:08.524129 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:57:08.524384 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:57:08.524197 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 19:58:12.524183098 +0000 UTC m=+161.148687113 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:57:41.544161 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:57:41.544106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:57:41.544738 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:57:41.544256 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 19:57:41.544738 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:57:41.544320 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 19:59:43.5443057 +0000 UTC m=+252.168809714 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : secret "metrics-daemon-secret" not found Sep 09 19:58:07.696223 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:58:07.696175 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ldsrh" podUID="22ffb8c6-7119-4d56-b048-6f5111a8a5cf" Sep 09 19:58:07.885171 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:58:07.885140 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 19:58:08.356062 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:08.356034 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ldsrh" Sep 09 19:58:12.551421 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:12.551382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 19:58:12.551906 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:58:12.551538 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 19:58:12.551906 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:58:12.551600 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls podName:22ffb8c6-7119-4d56-b048-6f5111a8a5cf nodeName:}" failed. No retries permitted until 2026-09-09 20:00:14.551582964 +0000 UTC m=+283.176087000 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls") pod "dns-default-ldsrh" (UID: "22ffb8c6-7119-4d56-b048-6f5111a8a5cf") : secret "dns-default-metrics-tls" not found Sep 09 19:58:21.875992 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:21.875961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:58:46.160171 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:46.160127 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" podUID="3dbb1538-5c31-46cd-b736-e32ef434b3bb" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Sep 09 19:58:46.446183 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:46.446150 2577 generic.go:356] "Generic (PLEG): container finished" podID="3dbb1538-5c31-46cd-b736-e32ef434b3bb" containerID="38c955c14ab731b628f32fac4474350c93796eedd0361dae12020a2ed58c1a9b" exitCode=1 Sep 09 19:58:46.446335 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:46.446223 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" event={"ID":"3dbb1538-5c31-46cd-b736-e32ef434b3bb","Type":"ContainerDied","Data":"38c955c14ab731b628f32fac4474350c93796eedd0361dae12020a2ed58c1a9b"} Sep 09 19:58:46.446573 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:46.446556 2577 scope.go:117] "RemoveContainer" containerID="38c955c14ab731b628f32fac4474350c93796eedd0361dae12020a2ed58c1a9b" Sep 09 19:58:47.450559 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:47.450527 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" event={"ID":"3dbb1538-5c31-46cd-b736-e32ef434b3bb","Type":"ContainerStarted","Data":"1583a81ea2b37d5ee7ab2bb425e21eb614096f30d2a9d709a2f8f250e9edaa8f"} Sep 09 19:58:47.451015 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:47.450802 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:58:47.451471 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:58:47.451439 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d59796848-qpkr4" Sep 09 19:59:33.432560 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.432522 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx"] Sep 09 19:59:33.435129 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.435114 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-mq4h4"] Sep 09 19:59:33.435275 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.435255 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.437660 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.437644 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.439732 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.439712 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 19:59:33.441214 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.441185 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 19:59:33.441698 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.441431 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kb84w\"" Sep 09 19:59:33.442026 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.442010 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 19:59:33.443179 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.443163 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 09 19:59:33.443389 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.443377 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 19:59:33.443739 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.443721 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nbqz7\"" Sep 09 19:59:33.443942 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.443928 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 19:59:33.444719 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.444700 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 19:59:33.451020 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.450997 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 09 19:59:33.452816 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.452801 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 09 19:59:33.470996 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.470975 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-mq4h4"] Sep 09 19:59:33.475561 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.475542 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx"] Sep 09 19:59:33.493034 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whtzv\" (UniqueName: \"kubernetes.io/projected/d9024083-10c8-42cc-94c4-4bf74db40a04-kube-api-access-whtzv\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.493194 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9024083-10c8-42cc-94c4-4bf74db40a04-serving-cert\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.493194 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.493275 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493213 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-tmp\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.493275 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.493347 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-snapshots\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.493347 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgxg2\" (UniqueName: \"kubernetes.io/projected/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-kube-api-access-rgxg2\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.493416 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d9024083-10c8-42cc-94c4-4bf74db40a04-config\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.493416 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.493376 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-serving-cert\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.524660 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.524632 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-k8npt"] Sep 09 19:59:33.527401 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.527387 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.529902 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.529883 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77b88cd7dd-sz5nj"] Sep 09 19:59:33.530371 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.530352 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 09 19:59:33.530488 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.530369 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 09 19:59:33.530628 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.530615 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ztfzv\"" Sep 09 19:59:33.531624 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.531600 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 09 19:59:33.532722 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.532705 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-779f6fdd5-chzk2"] Sep 09 19:59:33.532851 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.532834 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.533051 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.533028 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 09 19:59:33.535481 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.535396 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.535870 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.535850 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 09 19:59:33.535870 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.535861 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 09 19:59:33.536026 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.535994 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 09 19:59:33.536106 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.536087 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 09 19:59:33.536233 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.536107 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 19:59:33.536316 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.536284 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 19:59:33.537694 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.537526 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nqffj\"" Sep 09 19:59:33.539412 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.539396 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 19:59:33.540288 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.540270 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-k8npt"] Sep 09 19:59:33.544084 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.544061 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 19:59:33.544084 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.544079 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 19:59:33.544638 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.544622 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6tbbl\"" Sep 09 19:59:33.546042 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.546025 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 09 19:59:33.549138 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.549121 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 19:59:33.552881 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.552854 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77b88cd7dd-sz5nj"] Sep 09 19:59:33.553626 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.553608 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779f6fdd5-chzk2"] Sep 09 19:59:33.594554 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594531 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-tmp\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.594637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.594637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594576 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.594637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-trusted-ca\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.594637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594611 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-snapshots\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.594637 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594629 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.594842 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594672 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-serving-cert\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.594842 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgxg2\" (UniqueName: \"kubernetes.io/projected/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-kube-api-access-rgxg2\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.594842 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d9024083-10c8-42cc-94c4-4bf74db40a04-config\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.594842 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-serving-cert\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.595022 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-certificates\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.595022 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594887 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whtzv\" (UniqueName: \"kubernetes.io/projected/d9024083-10c8-42cc-94c4-4bf74db40a04-kube-api-access-whtzv\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.595022 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvsr9\" (UniqueName: \"kubernetes.io/projected/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-kube-api-access-gvsr9\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.595022 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-tmp\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.595022 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.594974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-bound-sa-token\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.595022 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-stats-auth\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.595022 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b35ab8c-d801-4bcf-96c7-25de0387cae1-ca-trust-extracted\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-installation-pull-secrets\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595100 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-default-certificate\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595175 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-image-registry-private-configuration\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595193 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595208 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595221 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq6g4\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-kube-api-access-qq6g4\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595267 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-snapshots\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595273 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-config\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7tph\" (UniqueName: \"kubernetes.io/projected/80f2786e-2c92-4c86-b816-0c0f9089b3d6-kube-api-access-w7tph\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.595360 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595335 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9024083-10c8-42cc-94c4-4bf74db40a04-serving-cert\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.595825 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-trusted-ca\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.595825 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.595825 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.595566 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d9024083-10c8-42cc-94c4-4bf74db40a04-config\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.596139 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.596122 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.597524 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.597505 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-serving-cert\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.597685 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.597670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9024083-10c8-42cc-94c4-4bf74db40a04-serving-cert\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.603747 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.603719 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whtzv\" (UniqueName: \"kubernetes.io/projected/d9024083-10c8-42cc-94c4-4bf74db40a04-kube-api-access-whtzv\") pod \"service-ca-operator-859f94c4b-qc4mx\" (UID: \"d9024083-10c8-42cc-94c4-4bf74db40a04\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.603889 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.603873 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgxg2\" (UniqueName: \"kubernetes.io/projected/bf40b6d8-3cdf-436f-b3c3-09fb46d7c383-kube-api-access-rgxg2\") pod \"insights-operator-76bcd6ccf5-mq4h4\" (UID: \"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.696313 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-image-registry-private-configuration\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.696313 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.696313 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq6g4\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-kube-api-access-qq6g4\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.696313 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-config\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.696313 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696316 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7tph\" (UniqueName: \"kubernetes.io/projected/80f2786e-2c92-4c86-b816-0c0f9089b3d6-kube-api-access-w7tph\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:33.696407 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:33.696428 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779f6fdd5-chzk2: secret "image-registry-tls" not found Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696471 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-trusted-ca\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696523 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:33.696555 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls podName:8b35ab8c-d801-4bcf-96c7-25de0387cae1 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:34.196531647 +0000 UTC m=+242.821035666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls") pod "image-registry-779f6fdd5-chzk2" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1") : secret "image-registry-tls" not found Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:33.696602 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:34.196588364 +0000 UTC m=+242.821092379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : configmap references non-existent config key: service-ca.crt Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-trusted-ca\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-serving-cert\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.696702 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-certificates\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:33.696761 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:33.696821 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:34.196804289 +0000 UTC m=+242.821308306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : secret "router-metrics-certs-default" not found Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.696951 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvsr9\" (UniqueName: \"kubernetes.io/projected/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-kube-api-access-gvsr9\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-bound-sa-token\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-stats-auth\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697066 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b35ab8c-d801-4bcf-96c7-25de0387cae1-ca-trust-extracted\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697069 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-config\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-installation-pull-secrets\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.697185 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697142 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-default-certificate\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.697654 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697353 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-certificates\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.697654 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.697477 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b35ab8c-d801-4bcf-96c7-25de0387cae1-ca-trust-extracted\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.698134 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.698108 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-trusted-ca\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.698236 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.698178 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-trusted-ca\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.699343 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.699315 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-image-registry-private-configuration\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.699726 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.699705 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-serving-cert\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.699826 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.699791 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-stats-auth\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.700170 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.700149 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-installation-pull-secrets\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.700595 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.700576 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-default-certificate\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.706433 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.706404 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq6g4\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-kube-api-access-qq6g4\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.706563 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.706492 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7tph\" (UniqueName: \"kubernetes.io/projected/80f2786e-2c92-4c86-b816-0c0f9089b3d6-kube-api-access-w7tph\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:33.707088 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.707068 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-bound-sa-token\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:33.707233 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.707213 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvsr9\" (UniqueName: \"kubernetes.io/projected/6a980afb-3ae2-491e-963e-fea2f4fa2ef2-kube-api-access-gvsr9\") pod \"console-operator-85b454874b-k8npt\" (UID: \"6a980afb-3ae2-491e-963e-fea2f4fa2ef2\") " pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:33.744549 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.744525 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" Sep 09 19:59:33.750825 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.750811 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" Sep 09 19:59:33.838615 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:33.838581 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:34.025343 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.025256 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx"] Sep 09 19:59:34.026293 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.026269 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-mq4h4"] Sep 09 19:59:34.028045 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:59:34.028019 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9024083_10c8_42cc_94c4_4bf74db40a04.slice/crio-41e72fd19541f1513130e211d28ca762a347b397237bb7ed97212f2b81a4df6b WatchSource:0}: Error finding container 41e72fd19541f1513130e211d28ca762a347b397237bb7ed97212f2b81a4df6b: Status 404 returned error can't find the container with id 41e72fd19541f1513130e211d28ca762a347b397237bb7ed97212f2b81a4df6b Sep 09 19:59:34.028432 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:59:34.028413 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf40b6d8_3cdf_436f_b3c3_09fb46d7c383.slice/crio-ac80a30d72637a28372e7e4917515844f106f4a4cf2d6978772c35b28636abe0 WatchSource:0}: Error finding container ac80a30d72637a28372e7e4917515844f106f4a4cf2d6978772c35b28636abe0: Status 404 returned error can't find the container with id ac80a30d72637a28372e7e4917515844f106f4a4cf2d6978772c35b28636abe0 Sep 09 19:59:34.109241 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.109205 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-k8npt"] Sep 09 19:59:34.114275 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:59:34.114248 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a980afb_3ae2_491e_963e_fea2f4fa2ef2.slice/crio-b6d8f604a7ca525c138a473534d37b34eb493f5dfcb052151ef610ce0ff6e254 WatchSource:0}: Error finding container b6d8f604a7ca525c138a473534d37b34eb493f5dfcb052151ef610ce0ff6e254: Status 404 returned error can't find the container with id b6d8f604a7ca525c138a473534d37b34eb493f5dfcb052151ef610ce0ff6e254 Sep 09 19:59:34.201986 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.201951 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:34.201986 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.201989 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:34.202195 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.202047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:34.202195 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:34.202115 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:35.20209715 +0000 UTC m=+243.826601166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : configmap references non-existent config key: service-ca.crt Sep 09 19:59:34.202195 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:34.202137 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 19:59:34.202195 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:34.202147 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779f6fdd5-chzk2: secret "image-registry-tls" not found Sep 09 19:59:34.202195 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:34.202182 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls podName:8b35ab8c-d801-4bcf-96c7-25de0387cae1 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:35.202171095 +0000 UTC m=+243.826675109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls") pod "image-registry-779f6fdd5-chzk2" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1") : secret "image-registry-tls" not found Sep 09 19:59:34.202195 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:34.202187 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 19:59:34.202418 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:34.202249 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:35.202233018 +0000 UTC m=+243.826737061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : secret "router-metrics-certs-default" not found Sep 09 19:59:34.567974 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.567942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" event={"ID":"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383","Type":"ContainerStarted","Data":"ac80a30d72637a28372e7e4917515844f106f4a4cf2d6978772c35b28636abe0"} Sep 09 19:59:34.569084 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.569065 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-k8npt" event={"ID":"6a980afb-3ae2-491e-963e-fea2f4fa2ef2","Type":"ContainerStarted","Data":"b6d8f604a7ca525c138a473534d37b34eb493f5dfcb052151ef610ce0ff6e254"} Sep 09 19:59:34.570118 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:34.570102 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" event={"ID":"d9024083-10c8-42cc-94c4-4bf74db40a04","Type":"ContainerStarted","Data":"41e72fd19541f1513130e211d28ca762a347b397237bb7ed97212f2b81a4df6b"} Sep 09 19:59:35.209583 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:35.209545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:35.209773 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:35.209678 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:35.209773 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:35.209720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:35.209773 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:35.209728 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 19:59:35.209935 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:35.209801 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:37.209783029 +0000 UTC m=+245.834287061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : secret "router-metrics-certs-default" not found Sep 09 19:59:35.209935 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:35.209823 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:37.20981314 +0000 UTC m=+245.834317155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : configmap references non-existent config key: service-ca.crt Sep 09 19:59:35.209935 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:35.209885 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 19:59:35.209935 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:35.209895 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779f6fdd5-chzk2: secret "image-registry-tls" not found Sep 09 19:59:35.209935 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:35.209925 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls podName:8b35ab8c-d801-4bcf-96c7-25de0387cae1 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:37.209915179 +0000 UTC m=+245.834419198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls") pod "image-registry-779f6fdd5-chzk2" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1") : secret "image-registry-tls" not found Sep 09 19:59:37.227580 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.227535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.227597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.227702 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:37.227734 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:41.227709866 +0000 UTC m=+249.852214091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : configmap references non-existent config key: service-ca.crt Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:37.227775 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:37.227784 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:37.227800 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779f6fdd5-chzk2: secret "image-registry-tls" not found Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:37.227839 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:41.227822281 +0000 UTC m=+249.852326296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : secret "router-metrics-certs-default" not found Sep 09 19:59:37.228042 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:37.227859 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls podName:8b35ab8c-d801-4bcf-96c7-25de0387cae1 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:41.227849516 +0000 UTC m=+249.852353531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls") pod "image-registry-779f6fdd5-chzk2" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1") : secret "image-registry-tls" not found Sep 09 19:59:37.580303 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.580229 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/0.log" Sep 09 19:59:37.580303 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.580276 2577 generic.go:356] "Generic (PLEG): container finished" podID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" containerID="860366a4d2032ed480d50ecfd6c7255cf5136b05073a9826539d1fd9f3882e8c" exitCode=255 Sep 09 19:59:37.580525 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.580367 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-k8npt" event={"ID":"6a980afb-3ae2-491e-963e-fea2f4fa2ef2","Type":"ContainerDied","Data":"860366a4d2032ed480d50ecfd6c7255cf5136b05073a9826539d1fd9f3882e8c"} Sep 09 19:59:37.580590 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.580564 2577 scope.go:117] "RemoveContainer" containerID="860366a4d2032ed480d50ecfd6c7255cf5136b05073a9826539d1fd9f3882e8c" Sep 09 19:59:37.582078 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.582053 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" event={"ID":"d9024083-10c8-42cc-94c4-4bf74db40a04","Type":"ContainerStarted","Data":"2f0934bef2b17b797b46a424fa9860b6c6235a9ee5958d7e76cab15937bd1a0a"} Sep 09 19:59:37.583699 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.583668 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" event={"ID":"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383","Type":"ContainerStarted","Data":"66c485c5f6652e8c59e13494eaf3b98d3a81bd0ca16f4c10558568108a1bc4c3"} Sep 09 19:59:37.619505 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.619465 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" podStartSLOduration=1.334663061 podStartE2EDuration="4.619434648s" totalImagesPullingTime="3.284771587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:59:37.618873314 +0000 UTC m=+246.243377354" watchObservedRunningTime="2026-09-09 19:59:37.619434648 +0000 UTC m=+246.243938722" Sep 09 19:59:37.640131 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:37.640081 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" podStartSLOduration=1.3525099329999999 podStartE2EDuration="4.640062688s" totalImagesPullingTime="3.287552755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:59:37.63865437 +0000 UTC m=+246.263158409" watchObservedRunningTime="2026-09-09 19:59:37.640062688 +0000 UTC m=+246.264566727" Sep 09 19:59:38.537691 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.537640 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-clrhp"] Sep 09 19:59:38.540797 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.540778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.543444 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.543426 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 19:59:38.543444 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.543424 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 19:59:38.545924 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.545906 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xxlzl\"" Sep 09 19:59:38.553577 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.553559 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-clrhp"] Sep 09 19:59:38.588929 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.588857 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/1.log" Sep 09 19:59:38.589465 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.589433 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/0.log" Sep 09 19:59:38.589553 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.589495 2577 generic.go:356] "Generic (PLEG): container finished" podID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" containerID="36604ec977f1c121d0cc07e8f03d03edb7f574a2bea567d1491785f811c572d6" exitCode=255 Sep 09 19:59:38.589600 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.589582 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-k8npt" event={"ID":"6a980afb-3ae2-491e-963e-fea2f4fa2ef2","Type":"ContainerDied","Data":"36604ec977f1c121d0cc07e8f03d03edb7f574a2bea567d1491785f811c572d6"} Sep 09 19:59:38.589651 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.589632 2577 scope.go:117] "RemoveContainer" containerID="860366a4d2032ed480d50ecfd6c7255cf5136b05073a9826539d1fd9f3882e8c" Sep 09 19:59:38.589993 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.589974 2577 scope.go:117] "RemoveContainer" containerID="36604ec977f1c121d0cc07e8f03d03edb7f574a2bea567d1491785f811c572d6" Sep 09 19:59:38.590171 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:38.590146 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-k8npt_openshift-console-operator(6a980afb-3ae2-491e-963e-fea2f4fa2ef2)\"" pod="openshift-console-operator/console-operator-85b454874b-k8npt" podUID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" Sep 09 19:59:38.641110 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.641088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kfr5\" (UniqueName: \"kubernetes.io/projected/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-api-access-7kfr5\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.641213 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.641128 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ba535aa-b22f-4e23-8617-971caabcb9c4-data-volume\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.641213 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.641156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.641213 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.641196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.641385 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.641374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ba535aa-b22f-4e23-8617-971caabcb9c4-crio-socket\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.742138 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.742111 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ba535aa-b22f-4e23-8617-971caabcb9c4-crio-socket\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.742237 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.742186 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kfr5\" (UniqueName: \"kubernetes.io/projected/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-api-access-7kfr5\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.742237 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.742217 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ba535aa-b22f-4e23-8617-971caabcb9c4-data-volume\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.742237 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.742220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ba535aa-b22f-4e23-8617-971caabcb9c4-crio-socket\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.742365 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.742240 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.742365 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.742276 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.742478 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:38.742383 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 19:59:38.742478 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:38.742384 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 19:59:38.742478 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:38.742442 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:39.242425353 +0000 UTC m=+247.866929375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : secret "insights-runtime-extractor-tls" not found Sep 09 19:59:38.742598 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:38.742480 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:39.24247004 +0000 UTC m=+247.866974061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : configmap "kube-rbac-proxy" not found Sep 09 19:59:38.742598 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.742491 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ba535aa-b22f-4e23-8617-971caabcb9c4-data-volume\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:38.751446 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:38.751418 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kfr5\" (UniqueName: \"kubernetes.io/projected/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-api-access-7kfr5\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:39.246628 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:39.246586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:39.246794 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:39.246693 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:39.246794 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:39.246742 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 19:59:39.246861 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:39.246810 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:40.246794246 +0000 UTC m=+248.871298260 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : secret "insights-runtime-extractor-tls" not found Sep 09 19:59:39.246861 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:39.246752 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 19:59:39.246928 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:39.246886 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:40.246873351 +0000 UTC m=+248.871377386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : configmap "kube-rbac-proxy" not found Sep 09 19:59:39.594857 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:39.594791 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/1.log" Sep 09 19:59:39.595195 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:39.595113 2577 scope.go:117] "RemoveContainer" containerID="36604ec977f1c121d0cc07e8f03d03edb7f574a2bea567d1491785f811c572d6" Sep 09 19:59:39.595273 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:39.595255 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-k8npt_openshift-console-operator(6a980afb-3ae2-491e-963e-fea2f4fa2ef2)\"" pod="openshift-console-operator/console-operator-85b454874b-k8npt" podUID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" Sep 09 19:59:40.253716 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:40.253665 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:40.253928 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:40.253737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:40.253928 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:40.253812 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 19:59:40.253928 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:40.253860 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 19:59:40.253928 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:40.253888 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:42.253868706 +0000 UTC m=+250.878372728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : configmap "kube-rbac-proxy" not found Sep 09 19:59:40.253928 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:40.253912 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:42.25389663 +0000 UTC m=+250.878400649 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : secret "insights-runtime-extractor-tls" not found Sep 09 19:59:41.228206 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.228172 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-jrt5j"] Sep 09 19:59:41.232423 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.232408 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.235407 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.235376 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 09 19:59:41.235407 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.235402 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 09 19:59:41.236574 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.236550 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-h9z6l\"" Sep 09 19:59:41.236722 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.236550 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 09 19:59:41.236722 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.236651 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 09 19:59:41.239434 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.239404 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-jrt5j"] Sep 09 19:59:41.262933 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.262912 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:41.263045 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.262949 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:41.263045 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.262975 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:41.263139 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:41.263081 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 19:59:41.263139 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:41.263108 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779f6fdd5-chzk2: secret "image-registry-tls" not found Sep 09 19:59:41.263139 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:41.263128 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:49.263108304 +0000 UTC m=+257.887612337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : configmap references non-existent config key: service-ca.crt Sep 09 19:59:41.263139 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:41.263085 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 19:59:41.263259 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:41.263149 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls podName:8b35ab8c-d801-4bcf-96c7-25de0387cae1 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:49.26313829 +0000 UTC m=+257.887642307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls") pod "image-registry-779f6fdd5-chzk2" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1") : secret "image-registry-tls" not found Sep 09 19:59:41.263259 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:41.263166 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs podName:80f2786e-2c92-4c86-b816-0c0f9089b3d6 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:49.26315655 +0000 UTC m=+257.887660565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs") pod "router-default-77b88cd7dd-sz5nj" (UID: "80f2786e-2c92-4c86-b816-0c0f9089b3d6") : secret "router-metrics-certs-default" not found Sep 09 19:59:41.364262 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.364230 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-signing-cabundle\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.364366 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.364278 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-signing-key\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.364366 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.364356 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqf9c\" (UniqueName: \"kubernetes.io/projected/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-kube-api-access-fqf9c\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.465247 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.465215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-signing-key\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.465392 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.465306 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqf9c\" (UniqueName: \"kubernetes.io/projected/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-kube-api-access-fqf9c\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.465392 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.465341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-signing-cabundle\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.465915 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.465898 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-signing-cabundle\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.467704 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.467688 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-signing-key\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.473739 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.473714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqf9c\" (UniqueName: \"kubernetes.io/projected/919ee670-3cf6-4bdc-b6b9-75f28e0d5f25-kube-api-access-fqf9c\") pod \"service-ca-75d99c484-jrt5j\" (UID: \"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25\") " pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.503998 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.503927 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkj8g_c34f0e58-b333-4941-b0fb-8d0ec6c461cd/dns-node-resolver/0.log" Sep 09 19:59:41.542232 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.542204 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-jrt5j" Sep 09 19:59:41.811797 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:41.811673 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-jrt5j"] Sep 09 19:59:41.814135 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:59:41.814108 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919ee670_3cf6_4bdc_b6b9_75f28e0d5f25.slice/crio-6c5a6920077ab3d3d4c050e47f290db92e99bd8cc4960508368782cdae3c495b WatchSource:0}: Error finding container 6c5a6920077ab3d3d4c050e47f290db92e99bd8cc4960508368782cdae3c495b: Status 404 returned error can't find the container with id 6c5a6920077ab3d3d4c050e47f290db92e99bd8cc4960508368782cdae3c495b Sep 09 19:59:42.272630 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:42.272594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:42.273072 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:42.272657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:42.273072 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:42.272713 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 19:59:42.273072 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:42.272773 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:46.272758405 +0000 UTC m=+254.897262448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : configmap "kube-rbac-proxy" not found Sep 09 19:59:42.273072 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:42.272841 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 19:59:42.273072 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:42.272909 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:46.272890821 +0000 UTC m=+254.897394856 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : secret "insights-runtime-extractor-tls" not found Sep 09 19:59:42.303674 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:42.303648 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mcc67_78ec08ab-1e3b-4efc-98ed-e659284b71b6/node-ca/0.log" Sep 09 19:59:42.606938 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:42.606857 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-jrt5j" event={"ID":"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25","Type":"ContainerStarted","Data":"a0db8b0f2b55b24bd1cb591763c2db71427123f7cabd48da852a2259ec17dced"} Sep 09 19:59:42.606938 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:42.606908 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-jrt5j" event={"ID":"919ee670-3cf6-4bdc-b6b9-75f28e0d5f25","Type":"ContainerStarted","Data":"6c5a6920077ab3d3d4c050e47f290db92e99bd8cc4960508368782cdae3c495b"} Sep 09 19:59:42.634731 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:42.634687 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-jrt5j" podStartSLOduration=1.634672574 podStartE2EDuration="1.634672574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:59:42.633637684 +0000 UTC m=+251.258141721" watchObservedRunningTime="2026-09-09 19:59:42.634672574 +0000 UTC m=+251.259176611" Sep 09 19:59:42.903697 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:42.903583 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-84qc6_0e1e15a7-89e0-4945-9a6b-1e1d97f9648a/serve-healthcheck-canary/0.log" Sep 09 19:59:43.584464 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:43.584419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 19:59:43.584921 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:43.584611 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 19:59:43.584921 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:43.584694 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs podName:379c5732-e859-4f78-895c-62bd2f25915b nodeName:}" failed. No retries permitted until 2026-09-09 20:01:45.584673791 +0000 UTC m=+374.209177812 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs") pod "network-metrics-daemon-98kdp" (UID: "379c5732-e859-4f78-895c-62bd2f25915b") : secret "metrics-daemon-secret" not found Sep 09 19:59:43.839716 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:43.839634 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:43.839716 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:43.839709 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 19:59:43.840015 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:43.840003 2577 scope.go:117] "RemoveContainer" containerID="36604ec977f1c121d0cc07e8f03d03edb7f574a2bea567d1491785f811c572d6" Sep 09 19:59:43.840153 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:43.840137 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-k8npt_openshift-console-operator(6a980afb-3ae2-491e-963e-fea2f4fa2ef2)\"" pod="openshift-console-operator/console-operator-85b454874b-k8npt" podUID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" Sep 09 19:59:46.307984 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:46.307942 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:46.308392 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:46.308000 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:46.308392 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:46.308071 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 19:59:46.308392 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:46.308105 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 19:59:46.308392 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:46.308134 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:54.308119615 +0000 UTC m=+262.932623630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : configmap "kube-rbac-proxy" not found Sep 09 19:59:46.308392 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:46.308148 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls podName:0ba535aa-b22f-4e23-8617-971caabcb9c4 nodeName:}" failed. No retries permitted until 2026-09-09 19:59:54.308142044 +0000 UTC m=+262.932646058 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-clrhp" (UID: "0ba535aa-b22f-4e23-8617-971caabcb9c4") : secret "insights-runtime-extractor-tls" not found Sep 09 19:59:49.333464 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.333411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:49.333464 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.333468 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:49.333990 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.333540 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:49.334115 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.334089 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80f2786e-2c92-4c86-b816-0c0f9089b3d6-service-ca-bundle\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:49.336093 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.336073 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"image-registry-779f6fdd5-chzk2\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:49.336174 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.336132 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2786e-2c92-4c86-b816-0c0f9089b3d6-metrics-certs\") pod \"router-default-77b88cd7dd-sz5nj\" (UID: \"80f2786e-2c92-4c86-b816-0c0f9089b3d6\") " pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:49.445015 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.444974 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:49.450249 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.450220 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:49.726898 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.726866 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77b88cd7dd-sz5nj"] Sep 09 19:59:49.730378 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:59:49.730350 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f2786e_2c92_4c86_b816_0c0f9089b3d6.slice/crio-36c6bee996b463152bc634338e217e295bcb563093b914ae1c464928bee17b6c WatchSource:0}: Error finding container 36c6bee996b463152bc634338e217e295bcb563093b914ae1c464928bee17b6c: Status 404 returned error can't find the container with id 36c6bee996b463152bc634338e217e295bcb563093b914ae1c464928bee17b6c Sep 09 19:59:49.733867 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:49.733541 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779f6fdd5-chzk2"] Sep 09 19:59:49.739446 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:59:49.739425 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b35ab8c_d801_4bcf_96c7_25de0387cae1.slice/crio-2fb14db74fd21e5a92c5d71901356473ba7521249f929112a4f336777b52f343 WatchSource:0}: Error finding container 2fb14db74fd21e5a92c5d71901356473ba7521249f929112a4f336777b52f343: Status 404 returned error can't find the container with id 2fb14db74fd21e5a92c5d71901356473ba7521249f929112a4f336777b52f343 Sep 09 19:59:50.637339 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:50.637292 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" event={"ID":"80f2786e-2c92-4c86-b816-0c0f9089b3d6","Type":"ContainerStarted","Data":"e11215dc95f109696aba9f4b7b0738ad6008413196ad006e068305fb9f1eae2f"} Sep 09 19:59:50.637801 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:50.637349 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" event={"ID":"80f2786e-2c92-4c86-b816-0c0f9089b3d6","Type":"ContainerStarted","Data":"36c6bee996b463152bc634338e217e295bcb563093b914ae1c464928bee17b6c"} Sep 09 19:59:50.638862 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:50.638835 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" event={"ID":"8b35ab8c-d801-4bcf-96c7-25de0387cae1","Type":"ContainerStarted","Data":"3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c"} Sep 09 19:59:50.638992 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:50.638873 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" event={"ID":"8b35ab8c-d801-4bcf-96c7-25de0387cae1","Type":"ContainerStarted","Data":"2fb14db74fd21e5a92c5d71901356473ba7521249f929112a4f336777b52f343"} Sep 09 19:59:50.638992 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:50.638964 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 19:59:50.662288 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:50.662237 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" podStartSLOduration=17.662223655 podStartE2EDuration="17.662223655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:59:50.661234282 +0000 UTC m=+259.285738321" watchObservedRunningTime="2026-09-09 19:59:50.662223655 +0000 UTC m=+259.286727744" Sep 09 19:59:50.681116 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:50.681069 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" podStartSLOduration=17.681056071 podStartE2EDuration="17.681056071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:59:50.680317039 +0000 UTC m=+259.304821078" watchObservedRunningTime="2026-09-09 19:59:50.681056071 +0000 UTC m=+259.305560108" Sep 09 19:59:51.445282 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:51.445252 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:51.447655 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:51.447631 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:51.642805 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:51.642761 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:51.643861 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:51.643840 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77b88cd7dd-sz5nj" Sep 09 19:59:54.373833 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:54.373762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:54.373833 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:54.373800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:54.374290 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:54.374268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ba535aa-b22f-4e23-8617-971caabcb9c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:54.376249 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:54.376232 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ba535aa-b22f-4e23-8617-971caabcb9c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clrhp\" (UID: \"0ba535aa-b22f-4e23-8617-971caabcb9c4\") " pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:54.449674 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:54.449647 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-clrhp" Sep 09 19:59:54.720045 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:54.720013 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-clrhp"] Sep 09 19:59:54.723161 ip-10-0-142-99 kubenswrapper[2577]: W0909 19:59:54.723136 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba535aa_b22f_4e23_8617_971caabcb9c4.slice/crio-ca99c2ccaa7ed0cca850cc364a5350f8ff3eaf3435f756151ec8a49655f3afa6 WatchSource:0}: Error finding container ca99c2ccaa7ed0cca850cc364a5350f8ff3eaf3435f756151ec8a49655f3afa6: Status 404 returned error can't find the container with id ca99c2ccaa7ed0cca850cc364a5350f8ff3eaf3435f756151ec8a49655f3afa6 Sep 09 19:59:55.657282 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:55.657253 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clrhp" event={"ID":"0ba535aa-b22f-4e23-8617-971caabcb9c4","Type":"ContainerStarted","Data":"3fe18c1c3046aee8c1a4d7cf3a6be19fa63efae9ae25a8945f6f884e9c795a0b"} Sep 09 19:59:55.657578 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:55.657303 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clrhp" event={"ID":"0ba535aa-b22f-4e23-8617-971caabcb9c4","Type":"ContainerStarted","Data":"c55192aefc00483f8a0d5c89c77f03cc83bf19aa84b03c0444fe7bcde59acfbc"} Sep 09 19:59:55.657578 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:55.657320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clrhp" event={"ID":"0ba535aa-b22f-4e23-8617-971caabcb9c4","Type":"ContainerStarted","Data":"ca99c2ccaa7ed0cca850cc364a5350f8ff3eaf3435f756151ec8a49655f3afa6"} Sep 09 19:59:57.666526 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:57.666487 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clrhp" event={"ID":"0ba535aa-b22f-4e23-8617-971caabcb9c4","Type":"ContainerStarted","Data":"eb46c31af4825995a6db0b62d9ffb94defc439fe120e6377e5af14c51c650941"} Sep 09 19:59:57.693315 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:57.693267 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-clrhp" podStartSLOduration=17.530564051 podStartE2EDuration="19.693252892s" totalImagesPullingTime="2.162688841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 19:59:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 19:59:57.692439195 +0000 UTC m=+266.316943227" watchObservedRunningTime="2026-09-09 19:59:57.693252892 +0000 UTC m=+266.317756958" Sep 09 19:59:57.876101 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:57.876066 2577 scope.go:117] "RemoveContainer" containerID="36604ec977f1c121d0cc07e8f03d03edb7f574a2bea567d1491785f811c572d6" Sep 09 19:59:58.671165 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:58.671137 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 19:59:58.671574 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:58.671558 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/1.log" Sep 09 19:59:58.671620 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:58.671591 2577 generic.go:356] "Generic (PLEG): container finished" podID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" containerID="5e2185bbd92792d3556ea4d930c3effb394545ae654ee53ba1726e31ab171c36" exitCode=255 Sep 09 19:59:58.671705 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:58.671683 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-k8npt" event={"ID":"6a980afb-3ae2-491e-963e-fea2f4fa2ef2","Type":"ContainerDied","Data":"5e2185bbd92792d3556ea4d930c3effb394545ae654ee53ba1726e31ab171c36"} Sep 09 19:59:58.671763 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:58.671749 2577 scope.go:117] "RemoveContainer" containerID="36604ec977f1c121d0cc07e8f03d03edb7f574a2bea567d1491785f811c572d6" Sep 09 19:59:58.672067 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:58.672051 2577 scope.go:117] "RemoveContainer" containerID="5e2185bbd92792d3556ea4d930c3effb394545ae654ee53ba1726e31ab171c36" Sep 09 19:59:58.672226 ip-10-0-142-99 kubenswrapper[2577]: E0909 19:59:58.672209 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-k8npt_openshift-console-operator(6a980afb-3ae2-491e-963e-fea2f4fa2ef2)\"" pod="openshift-console-operator/console-operator-85b454874b-k8npt" podUID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" Sep 09 19:59:59.676272 ip-10-0-142-99 kubenswrapper[2577]: I0909 19:59:59.676242 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:00:03.196235 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:03.196198 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-779f6fdd5-chzk2"] Sep 09 20:00:03.838880 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:03.838847 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 20:00:03.839068 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:03.838925 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 20:00:03.839214 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:03.839201 2577 scope.go:117] "RemoveContainer" containerID="5e2185bbd92792d3556ea4d930c3effb394545ae654ee53ba1726e31ab171c36" Sep 09 20:00:03.839361 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:00:03.839343 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-k8npt_openshift-console-operator(6a980afb-3ae2-491e-963e-fea2f4fa2ef2)\"" pod="openshift-console-operator/console-operator-85b454874b-k8npt" podUID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" Sep 09 20:00:11.357206 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:00:11.357163 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ldsrh" podUID="22ffb8c6-7119-4d56-b048-6f5111a8a5cf" Sep 09 20:00:11.715217 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:11.715182 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ldsrh" Sep 09 20:00:13.202548 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:13.202510 2577 patch_prober.go:28] interesting pod/image-registry-779f6fdd5-chzk2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 20:00:13.202901 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:13.202565 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" podUID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 20:00:14.628881 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:14.628832 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 20:00:14.631310 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:14.631284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22ffb8c6-7119-4d56-b048-6f5111a8a5cf-metrics-tls\") pod \"dns-default-ldsrh\" (UID: \"22ffb8c6-7119-4d56-b048-6f5111a8a5cf\") " pod="openshift-dns/dns-default-ldsrh" Sep 09 20:00:14.719371 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:14.719350 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4z7jm\"" Sep 09 20:00:14.727511 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:14.727493 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ldsrh" Sep 09 20:00:14.997881 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:14.997854 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ldsrh"] Sep 09 20:00:15.000562 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:00:15.000535 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ffb8c6_7119_4d56_b048_6f5111a8a5cf.slice/crio-c4e9da92356acc6b7b0fc7f3bb2b83af9b3a4f902cb6be05e62e2aa208b30755 WatchSource:0}: Error finding container c4e9da92356acc6b7b0fc7f3bb2b83af9b3a4f902cb6be05e62e2aa208b30755: Status 404 returned error can't find the container with id c4e9da92356acc6b7b0fc7f3bb2b83af9b3a4f902cb6be05e62e2aa208b30755 Sep 09 20:00:15.728054 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:15.728017 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ldsrh" event={"ID":"22ffb8c6-7119-4d56-b048-6f5111a8a5cf","Type":"ContainerStarted","Data":"c4e9da92356acc6b7b0fc7f3bb2b83af9b3a4f902cb6be05e62e2aa208b30755"} Sep 09 20:00:15.875419 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:15.875389 2577 scope.go:117] "RemoveContainer" containerID="5e2185bbd92792d3556ea4d930c3effb394545ae654ee53ba1726e31ab171c36" Sep 09 20:00:15.875639 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:00:15.875617 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-k8npt_openshift-console-operator(6a980afb-3ae2-491e-963e-fea2f4fa2ef2)\"" pod="openshift-console-operator/console-operator-85b454874b-k8npt" podUID="6a980afb-3ae2-491e-963e-fea2f4fa2ef2" Sep 09 20:00:17.735574 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:17.735538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ldsrh" event={"ID":"22ffb8c6-7119-4d56-b048-6f5111a8a5cf","Type":"ContainerStarted","Data":"ced76a409ba236dcc34e095478954e20c0a8b5e2e47bd81217ca4a66ef194d3a"} Sep 09 20:00:17.735947 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:17.735597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ldsrh" event={"ID":"22ffb8c6-7119-4d56-b048-6f5111a8a5cf","Type":"ContainerStarted","Data":"b2284cf664866378842f4e32e52204e8138b55de22d9005aca53b0cb3d92c521"} Sep 09 20:00:17.735947 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:17.735650 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ldsrh" Sep 09 20:00:17.756613 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:17.756564 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ldsrh" podStartSLOduration=251.435228071 podStartE2EDuration="4m13.756545873s" totalImagesPullingTime="2.321317802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:00:17.755734088 +0000 UTC m=+286.380238124" watchObservedRunningTime="2026-09-09 20:00:17.756545873 +0000 UTC m=+286.381049913" Sep 09 20:00:19.581540 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.581495 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m"] Sep 09 20:00:19.583819 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.583801 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.587548 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.587518 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 09 20:00:19.587692 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.587537 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 20:00:19.587756 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.587697 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 20:00:19.587812 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.587768 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 09 20:00:19.588677 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.588655 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 20:00:19.588807 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.588696 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ppk2j\"" Sep 09 20:00:19.598648 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.598626 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m"] Sep 09 20:00:19.607333 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.607312 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gkf5c"] Sep 09 20:00:19.609920 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.609904 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.612903 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.612885 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 20:00:19.612989 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.612903 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gpvzs\"" Sep 09 20:00:19.612989 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.612891 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 20:00:19.613261 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.613247 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 20:00:19.661487 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661443 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-tls\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661593 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661507 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-sys\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661593 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-textfile\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661720 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661606 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-wtmp\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661720 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg6k6\" (UniqueName: \"kubernetes.io/projected/295d5dc3-36b0-4ed9-bed8-814a46a842c4-kube-api-access-mg6k6\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661720 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-root\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8db9e7a-2268-44a2-a2cc-2393e524524d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.661811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8db9e7a-2268-44a2-a2cc-2393e524524d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.661811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661804 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/295d5dc3-36b0-4ed9-bed8-814a46a842c4-metrics-client-ca\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661901 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661842 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661901 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661869 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.661982 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8db9e7a-2268-44a2-a2cc-2393e524524d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.661982 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.661928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92tmn\" (UniqueName: \"kubernetes.io/projected/a8db9e7a-2268-44a2-a2cc-2393e524524d-kube-api-access-92tmn\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.763063 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763038 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-textfile\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763193 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-wtmp\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763193 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg6k6\" (UniqueName: \"kubernetes.io/projected/295d5dc3-36b0-4ed9-bed8-814a46a842c4-kube-api-access-mg6k6\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763193 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763152 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-root\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763193 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8db9e7a-2268-44a2-a2cc-2393e524524d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8db9e7a-2268-44a2-a2cc-2393e524524d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763227 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/295d5dc3-36b0-4ed9-bed8-814a46a842c4-metrics-client-ca\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763251 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-root\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8db9e7a-2268-44a2-a2cc-2393e524524d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763353 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-wtmp\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92tmn\" (UniqueName: \"kubernetes.io/projected/a8db9e7a-2268-44a2-a2cc-2393e524524d-kube-api-access-92tmn\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-textfile\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763416 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-tls\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763914 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763477 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-sys\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.763914 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763577 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/295d5dc3-36b0-4ed9-bed8-814a46a842c4-sys\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.764016 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763953 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/295d5dc3-36b0-4ed9-bed8-814a46a842c4-metrics-client-ca\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.764016 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.763968 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.764016 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.764006 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8db9e7a-2268-44a2-a2cc-2393e524524d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.765921 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.765893 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8db9e7a-2268-44a2-a2cc-2393e524524d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.766084 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.766067 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-tls\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.766150 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.766092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/295d5dc3-36b0-4ed9-bed8-814a46a842c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.766515 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.766496 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8db9e7a-2268-44a2-a2cc-2393e524524d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.771436 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.771418 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg6k6\" (UniqueName: \"kubernetes.io/projected/295d5dc3-36b0-4ed9-bed8-814a46a842c4-kube-api-access-mg6k6\") pod \"node-exporter-gkf5c\" (UID: \"295d5dc3-36b0-4ed9-bed8-814a46a842c4\") " pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.771678 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.771662 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92tmn\" (UniqueName: \"kubernetes.io/projected/a8db9e7a-2268-44a2-a2cc-2393e524524d-kube-api-access-92tmn\") pod \"openshift-state-metrics-657fd6c54-8pk5m\" (UID: \"a8db9e7a-2268-44a2-a2cc-2393e524524d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.895251 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.895181 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" Sep 09 20:00:19.920508 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:19.920483 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gkf5c" Sep 09 20:00:19.930086 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:00:19.930047 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295d5dc3_36b0_4ed9_bed8_814a46a842c4.slice/crio-08f39492890fd2bac54e937c07b4693c7a37591bf7d0a3a3b5fb2c65bae59ec5 WatchSource:0}: Error finding container 08f39492890fd2bac54e937c07b4693c7a37591bf7d0a3a3b5fb2c65bae59ec5: Status 404 returned error can't find the container with id 08f39492890fd2bac54e937c07b4693c7a37591bf7d0a3a3b5fb2c65bae59ec5 Sep 09 20:00:20.171329 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:20.171302 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m"] Sep 09 20:00:20.174130 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:00:20.174105 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8db9e7a_2268_44a2_a2cc_2393e524524d.slice/crio-8d098c7dcf2aa320179109f8b4ecd2416af08ee38422182943d2193b5f462b2b WatchSource:0}: Error finding container 8d098c7dcf2aa320179109f8b4ecd2416af08ee38422182943d2193b5f462b2b: Status 404 returned error can't find the container with id 8d098c7dcf2aa320179109f8b4ecd2416af08ee38422182943d2193b5f462b2b Sep 09 20:00:20.747130 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:20.747086 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" event={"ID":"a8db9e7a-2268-44a2-a2cc-2393e524524d","Type":"ContainerStarted","Data":"ce0b4cc1154e7e80ea61f38f3334940d1e8412e20e2e0afd09936feaf2bd3381"} Sep 09 20:00:20.747594 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:20.747159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" event={"ID":"a8db9e7a-2268-44a2-a2cc-2393e524524d","Type":"ContainerStarted","Data":"74dcd9865820cbc4ad6f42442ef6e06e2671e933312100669c90876ac093b5c8"} Sep 09 20:00:20.747594 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:20.747184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" event={"ID":"a8db9e7a-2268-44a2-a2cc-2393e524524d","Type":"ContainerStarted","Data":"8d098c7dcf2aa320179109f8b4ecd2416af08ee38422182943d2193b5f462b2b"} Sep 09 20:00:20.748327 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:20.748305 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkf5c" event={"ID":"295d5dc3-36b0-4ed9-bed8-814a46a842c4","Type":"ContainerStarted","Data":"08f39492890fd2bac54e937c07b4693c7a37591bf7d0a3a3b5fb2c65bae59ec5"} Sep 09 20:00:21.573405 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.573365 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q"] Sep 09 20:00:21.576329 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.576305 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.579338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.579130 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 09 20:00:21.579338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.579154 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-crfhf\"" Sep 09 20:00:21.579338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.579184 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 09 20:00:21.579338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.579197 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 09 20:00:21.579338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.579157 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 09 20:00:21.579338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.579282 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 09 20:00:21.579673 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.579559 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fnv6h0bhmv8uc\"" Sep 09 20:00:21.587385 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.587362 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q"] Sep 09 20:00:21.684189 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.684519 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684383 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.684597 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-metrics-client-ca\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.684659 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684628 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.684708 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-grpc-tls\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.684759 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684707 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b58h\" (UniqueName: \"kubernetes.io/projected/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-kube-api-access-5b58h\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.684759 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-tls\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.684853 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.684764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786080 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786080 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786606 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786606 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-metrics-client-ca\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786606 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786276 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786606 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786326 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-grpc-tls\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786606 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786364 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b58h\" (UniqueName: \"kubernetes.io/projected/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-kube-api-access-5b58h\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786606 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-tls\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.786935 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.786913 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-metrics-client-ca\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.788828 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.788794 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.789137 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.789118 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.789137 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.789148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.789262 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.789234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.789765 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.789737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-thanos-querier-tls\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.790815 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.790781 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-secret-grpc-tls\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.795188 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.795167 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b58h\" (UniqueName: \"kubernetes.io/projected/a731fbbf-ef3c-4532-8beb-f2f2adafa3bb-kube-api-access-5b58h\") pod \"thanos-querier-6d7b76fdd4-mfx9q\" (UID: \"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb\") " pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:21.887271 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:21.887193 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:22.164159 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:22.164086 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q"] Sep 09 20:00:22.168131 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:00:22.168104 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda731fbbf_ef3c_4532_8beb_f2f2adafa3bb.slice/crio-a19cdc37799e98a5f609babba30f3ac160fb5ac1f7eb7f17ff0079430973bedb WatchSource:0}: Error finding container a19cdc37799e98a5f609babba30f3ac160fb5ac1f7eb7f17ff0079430973bedb: Status 404 returned error can't find the container with id a19cdc37799e98a5f609babba30f3ac160fb5ac1f7eb7f17ff0079430973bedb Sep 09 20:00:22.759844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:22.759809 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" event={"ID":"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb","Type":"ContainerStarted","Data":"a19cdc37799e98a5f609babba30f3ac160fb5ac1f7eb7f17ff0079430973bedb"} Sep 09 20:00:22.762117 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:22.762078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" event={"ID":"a8db9e7a-2268-44a2-a2cc-2393e524524d","Type":"ContainerStarted","Data":"e28453ed504fdddbaf63e3c9334e2416fa10c50435b6286ab36f2604919daf4e"} Sep 09 20:00:22.782833 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:22.782778 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8pk5m" podStartSLOduration=2.353949057 podStartE2EDuration="3.782761815s" totalImagesPullingTime="1.428812758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:00:22.78163148 +0000 UTC m=+291.406135509" watchObservedRunningTime="2026-09-09 20:00:22.782761815 +0000 UTC m=+291.407265851" Sep 09 20:00:23.201141 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:23.201110 2577 patch_prober.go:28] interesting pod/image-registry-779f6fdd5-chzk2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 20:00:23.201417 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:23.201157 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" podUID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 20:00:23.768338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:23.768304 2577 generic.go:356] "Generic (PLEG): container finished" podID="295d5dc3-36b0-4ed9-bed8-814a46a842c4" containerID="31131256a95bbf4769e3d13976028f2906135ddda16050003e6bf3bbff98724c" exitCode=0 Sep 09 20:00:23.768523 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:23.768396 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkf5c" event={"ID":"295d5dc3-36b0-4ed9-bed8-814a46a842c4","Type":"ContainerDied","Data":"31131256a95bbf4769e3d13976028f2906135ddda16050003e6bf3bbff98724c"} Sep 09 20:00:24.775863 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:24.775826 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkf5c" event={"ID":"295d5dc3-36b0-4ed9-bed8-814a46a842c4","Type":"ContainerStarted","Data":"43b7645b59c16d1c24544cb04424e20aa3d286e458501a05cca961a1a14ad922"} Sep 09 20:00:24.776309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:24.775887 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkf5c" event={"ID":"295d5dc3-36b0-4ed9-bed8-814a46a842c4","Type":"ContainerStarted","Data":"8459f52ed456b96fde3f8ec2a9f9f5403f0ed0edd04e24abe86b2bb17cb13446"} Sep 09 20:00:24.802635 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:24.800163 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gkf5c" podStartSLOduration=2.606562131 podStartE2EDuration="5.800143733s" totalImagesPullingTime="3.124702114s" totalInitContainerRuntime="68.879488ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:00:24.798684323 +0000 UTC m=+293.423188360" watchObservedRunningTime="2026-09-09 20:00:24.800143733 +0000 UTC m=+293.424647771" Sep 09 20:00:24.877433 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:00:24.877393 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-98kdp" podUID="379c5732-e859-4f78-895c-62bd2f25915b" Sep 09 20:00:25.781386 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.781352 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" event={"ID":"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb","Type":"ContainerStarted","Data":"9abfbcff1428ec4509e3406e80ae1ec63d8eb43445c2ee67888ea927baf57461"} Sep 09 20:00:25.781757 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.781445 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" event={"ID":"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb","Type":"ContainerStarted","Data":"3c7a0932676520f1726a2554d275ef063d178e3fd789bcd4b98e28a4fe88f11b"} Sep 09 20:00:25.781757 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.781492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" event={"ID":"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb","Type":"ContainerStarted","Data":"baaeb3b0a4f4f2b038f56381b8bae58251896a94370b6a20aea866bd8d9c5f37"} Sep 09 20:00:25.861760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.861730 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:00:25.864507 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.864491 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.867703 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.867671 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 20:00:25.867844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.867678 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 20:00:25.867844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.867679 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 20:00:25.867844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.867805 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8c2llilpmuoul\"" Sep 09 20:00:25.867844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.867826 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 20:00:25.868030 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.867858 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 20:00:25.869123 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.869099 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 20:00:25.869244 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.869143 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 20:00:25.869244 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.869218 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 20:00:25.869338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.869258 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 20:00:25.869478 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.869444 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 20:00:25.869538 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.869529 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pw576\"" Sep 09 20:00:25.870193 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.870173 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 20:00:25.870287 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.870170 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 20:00:25.870287 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.870253 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 20:00:25.872225 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.872209 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 20:00:25.874598 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.874581 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 20:00:25.881520 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.881499 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:00:25.920084 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920194 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920194 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920194 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920194 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920320 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920320 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920228 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920320 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920408 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920318 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config-out\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920408 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920408 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920408 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920408 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920573 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920430 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920573 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920469 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920573 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czghb\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-kube-api-access-czghb\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920573 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920573 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920552 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920715 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920576 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:25.920715 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:25.920606 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-web-config\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021213 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021189 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021369 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021369 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021369 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021265 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config-out\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021369 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021369 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021369 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021345 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021369 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021365 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021473 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czghb\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-kube-api-access-czghb\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021494 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021524 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021568 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-web-config\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.021727 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.022476 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.021840 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.022857 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.022537 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.023490 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.023445 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.024596 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.024573 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.024724 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.024703 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.025016 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.024892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.025113 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.025024 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.025339 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.025234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config-out\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.026220 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.025525 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-web-config\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.026220 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.026009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.026220 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.026181 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.026863 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.026842 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.026994 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.026899 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.027091 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.027066 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.027091 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.026954 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.027327 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.027306 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.027381 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.027360 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.027708 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.027689 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.029253 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.029236 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.033035 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.032991 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czghb\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-kube-api-access-czghb\") pod \"prometheus-k8s-0\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.174125 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.174092 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:26.457019 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.456986 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:00:26.459837 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:00:26.459814 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa1e5d4_ae78_4ef5_a3e3_0eaee4855601.slice/crio-41621ed7a4e7afc505f7951ccfd4520c1d44f896826082e0cc48f3e9a6d877c3 WatchSource:0}: Error finding container 41621ed7a4e7afc505f7951ccfd4520c1d44f896826082e0cc48f3e9a6d877c3: Status 404 returned error can't find the container with id 41621ed7a4e7afc505f7951ccfd4520c1d44f896826082e0cc48f3e9a6d877c3 Sep 09 20:00:26.786411 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:26.786326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerStarted","Data":"41621ed7a4e7afc505f7951ccfd4520c1d44f896826082e0cc48f3e9a6d877c3"} Sep 09 20:00:27.740537 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:27.740508 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ldsrh" Sep 09 20:00:27.794381 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:27.794344 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" event={"ID":"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb","Type":"ContainerStarted","Data":"144fd03c16f7c67e6b281a6fec67e998c4cd754e951bf48d9d905a011b7bfe3a"} Sep 09 20:00:27.794772 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:27.794407 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" event={"ID":"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb","Type":"ContainerStarted","Data":"a55597292b75c654e4df5f7b6d471cf7aa0b0c3ab5d45eec056c3449fc346ff9"} Sep 09 20:00:27.794772 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:27.794432 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" event={"ID":"a731fbbf-ef3c-4532-8beb-f2f2adafa3bb","Type":"ContainerStarted","Data":"be25f493bbf19347eb8dfe997992e548d6b62ac84107c736692d2b7f57cd864a"} Sep 09 20:00:27.817665 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:27.817615 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" podStartSLOduration=2.166782388 podStartE2EDuration="6.817600138s" totalImagesPullingTime="4.65081775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:00:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:00:27.817398925 +0000 UTC m=+296.441902963" watchObservedRunningTime="2026-09-09 20:00:27.817600138 +0000 UTC m=+296.442104323" Sep 09 20:00:28.221122 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.221086 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" podUID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" containerName="registry" containerID="cri-o://3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c" gracePeriod=30 Sep 09 20:00:28.461082 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.461062 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 20:00:28.545146 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545070 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq6g4\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-kube-api-access-qq6g4\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545146 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545105 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545146 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545137 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-image-registry-private-configuration\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545365 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545155 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-installation-pull-secrets\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545365 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545185 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b35ab8c-d801-4bcf-96c7-25de0387cae1-ca-trust-extracted\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545365 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545225 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-trusted-ca\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545365 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545252 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-certificates\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545583 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545552 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-bound-sa-token\") pod \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\" (UID: \"8b35ab8c-d801-4bcf-96c7-25de0387cae1\") " Sep 09 20:00:28.545853 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545704 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:00:28.546152 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.545959 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:00:28.547701 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.547679 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:00:28.547872 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.547847 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:00:28.547963 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.547935 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-kube-api-access-qq6g4" (OuterVolumeSpecName: "kube-api-access-qq6g4") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "kube-api-access-qq6g4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:00:28.548019 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.547990 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:00:28.548162 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.548141 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:00:28.554106 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.554082 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b35ab8c-d801-4bcf-96c7-25de0387cae1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8b35ab8c-d801-4bcf-96c7-25de0387cae1" (UID: "8b35ab8c-d801-4bcf-96c7-25de0387cae1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:00:28.646698 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646664 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq6g4\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-kube-api-access-qq6g4\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.646698 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646696 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.646698 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646707 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-image-registry-private-configuration\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.646854 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646717 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b35ab8c-d801-4bcf-96c7-25de0387cae1-installation-pull-secrets\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.646854 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646725 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b35ab8c-d801-4bcf-96c7-25de0387cae1-ca-trust-extracted\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.646854 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646733 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-trusted-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.646854 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646741 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b35ab8c-d801-4bcf-96c7-25de0387cae1-registry-certificates\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.646854 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.646749 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b35ab8c-d801-4bcf-96c7-25de0387cae1-bound-sa-token\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:00:28.799914 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.799851 2577 generic.go:356] "Generic (PLEG): container finished" podID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" containerID="3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c" exitCode=0 Sep 09 20:00:28.799914 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.799901 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" Sep 09 20:00:28.800303 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.799946 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" event={"ID":"8b35ab8c-d801-4bcf-96c7-25de0387cae1","Type":"ContainerDied","Data":"3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c"} Sep 09 20:00:28.800303 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.799997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779f6fdd5-chzk2" event={"ID":"8b35ab8c-d801-4bcf-96c7-25de0387cae1","Type":"ContainerDied","Data":"2fb14db74fd21e5a92c5d71901356473ba7521249f929112a4f336777b52f343"} Sep 09 20:00:28.800303 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.800024 2577 scope.go:117] "RemoveContainer" containerID="3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c" Sep 09 20:00:28.800474 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.800435 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:28.808800 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.808783 2577 scope.go:117] "RemoveContainer" containerID="3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c" Sep 09 20:00:28.809098 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:00:28.809075 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c\": container with ID starting with 3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c not found: ID does not exist" containerID="3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c" Sep 09 20:00:28.809162 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.809109 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c"} err="failed to get container status \"3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c\": rpc error: code = NotFound desc = could not find container \"3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c\": container with ID starting with 3a4f1b70446299ac65912a2192b881029d844056a8692fcbf7abbe55f25a603c not found: ID does not exist" Sep 09 20:00:28.822412 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.822390 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-779f6fdd5-chzk2"] Sep 09 20:00:28.826203 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.826185 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-779f6fdd5-chzk2"] Sep 09 20:00:28.875648 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:28.875622 2577 scope.go:117] "RemoveContainer" containerID="5e2185bbd92792d3556ea4d930c3effb394545ae654ee53ba1726e31ab171c36" Sep 09 20:00:29.807721 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:29.807669 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:00:29.808143 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:29.807834 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-k8npt" event={"ID":"6a980afb-3ae2-491e-963e-fea2f4fa2ef2","Type":"ContainerStarted","Data":"ef1e04d6e949fffb8a64b983149f63bb8c0855858a33f82c0408d677911e69ef"} Sep 09 20:00:29.808499 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:29.808479 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 20:00:29.813635 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:29.813611 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-k8npt" Sep 09 20:00:29.828698 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:29.828648 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-k8npt" podStartSLOduration=53.633168305 podStartE2EDuration="56.828629571s" totalImagesPullingTime="3.195461266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:00:29.827330536 +0000 UTC m=+298.451834575" watchObservedRunningTime="2026-09-09 20:00:29.828629571 +0000 UTC m=+298.453133608" Sep 09 20:00:29.890923 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:29.890888 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" path="/var/lib/kubelet/pods/8b35ab8c-d801-4bcf-96c7-25de0387cae1/volumes" Sep 09 20:00:30.813017 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:30.812987 2577 generic.go:356] "Generic (PLEG): container finished" podID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" exitCode=0 Sep 09 20:00:30.813422 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:30.813084 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187"} Sep 09 20:00:31.766009 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:31.765984 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:00:31.767540 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:31.767440 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:00:31.781965 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:31.781945 2577 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 20:00:34.815981 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.815955 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d7b76fdd4-mfx9q" Sep 09 20:00:34.831444 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.831419 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerStarted","Data":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} Sep 09 20:00:34.831598 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.831497 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerStarted","Data":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} Sep 09 20:00:34.831598 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.831524 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerStarted","Data":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} Sep 09 20:00:34.831598 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.831548 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerStarted","Data":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} Sep 09 20:00:34.831598 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.831575 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerStarted","Data":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} Sep 09 20:00:34.831598 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.831599 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerStarted","Data":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} Sep 09 20:00:34.875145 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:34.875076 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.217389518 podStartE2EDuration="9.875053179s" totalImagesPullingTime="6.635384074s" totalInitContainerRuntime="22.279587ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:00:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:00:34.871903727 +0000 UTC m=+303.496407763" watchObservedRunningTime="2026-09-09 20:00:34.875053179 +0000 UTC m=+303.499557273" Sep 09 20:00:36.174785 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:36.174752 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:00:38.875869 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:38.875840 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 20:00:43.869039 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:43.869004 2577 generic.go:356] "Generic (PLEG): container finished" podID="d9024083-10c8-42cc-94c4-4bf74db40a04" containerID="2f0934bef2b17b797b46a424fa9860b6c6235a9ee5958d7e76cab15937bd1a0a" exitCode=0 Sep 09 20:00:43.869418 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:43.869076 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" event={"ID":"d9024083-10c8-42cc-94c4-4bf74db40a04","Type":"ContainerDied","Data":"2f0934bef2b17b797b46a424fa9860b6c6235a9ee5958d7e76cab15937bd1a0a"} Sep 09 20:00:43.869418 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:43.869373 2577 scope.go:117] "RemoveContainer" containerID="2f0934bef2b17b797b46a424fa9860b6c6235a9ee5958d7e76cab15937bd1a0a" Sep 09 20:00:44.874414 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:00:44.874378 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qc4mx" event={"ID":"d9024083-10c8-42cc-94c4-4bf74db40a04","Type":"ContainerStarted","Data":"c18da7c693702c03bfde7b33d6cdde025051546e3ae0ff910da64d7e69d0ed45"} Sep 09 20:01:12.976065 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:12.976035 2577 generic.go:356] "Generic (PLEG): container finished" podID="bf40b6d8-3cdf-436f-b3c3-09fb46d7c383" containerID="66c485c5f6652e8c59e13494eaf3b98d3a81bd0ca16f4c10558568108a1bc4c3" exitCode=0 Sep 09 20:01:12.976490 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:12.976092 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" event={"ID":"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383","Type":"ContainerDied","Data":"66c485c5f6652e8c59e13494eaf3b98d3a81bd0ca16f4c10558568108a1bc4c3"} Sep 09 20:01:12.976490 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:12.976441 2577 scope.go:117] "RemoveContainer" containerID="66c485c5f6652e8c59e13494eaf3b98d3a81bd0ca16f4c10558568108a1bc4c3" Sep 09 20:01:13.982146 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:13.982116 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mq4h4" event={"ID":"bf40b6d8-3cdf-436f-b3c3-09fb46d7c383","Type":"ContainerStarted","Data":"2c4e960e9595dce8615591e69d5880f5683d32472539c810f7f032ee821b485e"} Sep 09 20:01:26.175133 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:26.175041 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:26.191091 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:26.191056 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:27.044958 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:27.044932 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:44.243086 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.243053 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:01:44.243599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.243365 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="prometheus" containerID="cri-o://8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" gracePeriod=600 Sep 09 20:01:44.243599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.243380 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy" containerID="cri-o://bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" gracePeriod=600 Sep 09 20:01:44.243599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.243418 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="thanos-sidecar" containerID="cri-o://08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" gracePeriod=600 Sep 09 20:01:44.243599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.243438 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-web" containerID="cri-o://f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" gracePeriod=600 Sep 09 20:01:44.243599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.243489 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" gracePeriod=600 Sep 09 20:01:44.243599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.243431 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="config-reloader" containerID="cri-o://c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" gracePeriod=600 Sep 09 20:01:44.475716 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.475691 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:44.585661 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585584 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.585661 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585616 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-2\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.585661 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585647 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-kubelet-serving-ca-bundle\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.585920 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585676 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-kube-rbac-proxy\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.585920 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585846 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-serving-certs-ca-bundle\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.585920 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585909 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czghb\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-kube-api-access-czghb\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586075 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585940 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586075 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585967 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-metrics-client-certs\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586075 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585977 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:01:44.586075 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.585995 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-thanos-prometheus-http-client-file\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586075 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586036 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-trusted-ca-bundle\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586075 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586074 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-grpc-tls\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586100 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586112 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586129 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-1\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586148 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586165 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-tls-assets\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586201 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-web-config\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586243 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-0\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586356 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586283 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-db\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586498 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-metrics-client-ca\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586527 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config-out\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586553 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-tls\") pod \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\" (UID: \"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601\") " Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586586 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586734 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586843 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586862 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586878 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586893 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.586940 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.586907 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.588407 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.588352 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:01:44.589307 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.589280 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:01:44.589307 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.589287 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.589498 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.589331 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.589498 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.589368 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.589498 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.589401 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.590290 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.590185 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.590390 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.590343 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:01:44.590390 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.590359 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.590537 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.590487 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.591187 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.591166 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:01:44.591506 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.591488 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-kube-api-access-czghb" (OuterVolumeSpecName: "kube-api-access-czghb") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "kube-api-access-czghb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:01:44.591607 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.591515 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config-out" (OuterVolumeSpecName: "config-out") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:01:44.591649 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.591628 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config" (OuterVolumeSpecName: "config") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.601128 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.601110 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-web-config" (OuterVolumeSpecName: "web-config") pod "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" (UID: "6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:01:44.687601 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687576 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687601 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687596 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-grpc-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687601 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687606 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687615 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-tls-assets\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687623 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-web-config\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687630 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687638 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-prometheus-k8s-db\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687645 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-configmap-metrics-client-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687653 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config-out\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687661 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687669 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687677 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-kube-rbac-proxy\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687684 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czghb\" (UniqueName: \"kubernetes.io/projected/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-kube-api-access-czghb\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687692 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-config\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:44.687769 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:44.687699 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601-secret-metrics-client-certs\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:01:45.102862 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102827 2577 generic.go:356] "Generic (PLEG): container finished" podID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" exitCode=0 Sep 09 20:01:45.102862 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102852 2577 generic.go:356] "Generic (PLEG): container finished" podID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" exitCode=0 Sep 09 20:01:45.102862 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102857 2577 generic.go:356] "Generic (PLEG): container finished" podID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" exitCode=0 Sep 09 20:01:45.102862 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102863 2577 generic.go:356] "Generic (PLEG): container finished" podID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" exitCode=0 Sep 09 20:01:45.102862 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102868 2577 generic.go:356] "Generic (PLEG): container finished" podID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" exitCode=0 Sep 09 20:01:45.102862 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102873 2577 generic.go:356] "Generic (PLEG): container finished" podID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" exitCode=0 Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102899 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102914 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102957 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.102984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.103008 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.103032 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.103034 2577 scope.go:117] "RemoveContainer" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.103055 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} Sep 09 20:01:45.103309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.103077 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601","Type":"ContainerDied","Data":"41621ed7a4e7afc505f7951ccfd4520c1d44f896826082e0cc48f3e9a6d877c3"} Sep 09 20:01:45.111665 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.111650 2577 scope.go:117] "RemoveContainer" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.120754 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.120734 2577 scope.go:117] "RemoveContainer" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.127982 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.127958 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:01:45.128560 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.128546 2577 scope.go:117] "RemoveContainer" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.132121 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.132097 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:01:45.135606 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.135589 2577 scope.go:117] "RemoveContainer" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.142266 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.142252 2577 scope.go:117] "RemoveContainer" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.149093 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.149067 2577 scope.go:117] "RemoveContainer" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.155753 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.155733 2577 scope.go:117] "RemoveContainer" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.156034 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:01:45.156016 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": container with ID starting with bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04 not found: ID does not exist" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.156099 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156042 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} err="failed to get container status \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": rpc error: code = NotFound desc = could not find container \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": container with ID starting with bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04 not found: ID does not exist" Sep 09 20:01:45.156099 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156060 2577 scope.go:117] "RemoveContainer" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.156304 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:01:45.156289 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": container with ID starting with bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec not found: ID does not exist" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.156341 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156309 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} err="failed to get container status \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": rpc error: code = NotFound desc = could not find container \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": container with ID starting with bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec not found: ID does not exist" Sep 09 20:01:45.156341 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156323 2577 scope.go:117] "RemoveContainer" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.156617 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:01:45.156597 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": container with ID starting with f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098 not found: ID does not exist" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.156677 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156621 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} err="failed to get container status \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": rpc error: code = NotFound desc = could not find container \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": container with ID starting with f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098 not found: ID does not exist" Sep 09 20:01:45.156677 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156635 2577 scope.go:117] "RemoveContainer" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.156890 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:01:45.156874 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": container with ID starting with 08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c not found: ID does not exist" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.156936 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156893 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} err="failed to get container status \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": rpc error: code = NotFound desc = could not find container \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": container with ID starting with 08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c not found: ID does not exist" Sep 09 20:01:45.156936 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.156909 2577 scope.go:117] "RemoveContainer" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.157152 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:01:45.157134 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": container with ID starting with c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5 not found: ID does not exist" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.157217 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.157156 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} err="failed to get container status \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": rpc error: code = NotFound desc = could not find container \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": container with ID starting with c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5 not found: ID does not exist" Sep 09 20:01:45.157217 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.157171 2577 scope.go:117] "RemoveContainer" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.157445 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:01:45.157385 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": container with ID starting with 8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87 not found: ID does not exist" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.157445 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.157405 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} err="failed to get container status \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": rpc error: code = NotFound desc = could not find container \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": container with ID starting with 8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87 not found: ID does not exist" Sep 09 20:01:45.157445 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.157418 2577 scope.go:117] "RemoveContainer" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.157826 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:01:45.157807 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": container with ID starting with 02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187 not found: ID does not exist" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.157913 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.157833 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187"} err="failed to get container status \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": rpc error: code = NotFound desc = could not find container \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": container with ID starting with 02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187 not found: ID does not exist" Sep 09 20:01:45.157913 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.157854 2577 scope.go:117] "RemoveContainer" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.158147 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158130 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} err="failed to get container status \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": rpc error: code = NotFound desc = could not find container \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": container with ID starting with bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04 not found: ID does not exist" Sep 09 20:01:45.158214 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158146 2577 scope.go:117] "RemoveContainer" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.158392 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158377 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} err="failed to get container status \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": rpc error: code = NotFound desc = could not find container \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": container with ID starting with bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec not found: ID does not exist" Sep 09 20:01:45.158477 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158392 2577 scope.go:117] "RemoveContainer" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.158645 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158626 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} err="failed to get container status \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": rpc error: code = NotFound desc = could not find container \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": container with ID starting with f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098 not found: ID does not exist" Sep 09 20:01:45.158703 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158647 2577 scope.go:117] "RemoveContainer" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.158902 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158882 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} err="failed to get container status \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": rpc error: code = NotFound desc = could not find container \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": container with ID starting with 08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c not found: ID does not exist" Sep 09 20:01:45.158964 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.158903 2577 scope.go:117] "RemoveContainer" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.159133 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159117 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} err="failed to get container status \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": rpc error: code = NotFound desc = could not find container \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": container with ID starting with c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5 not found: ID does not exist" Sep 09 20:01:45.159237 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159133 2577 scope.go:117] "RemoveContainer" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.159353 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159335 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} err="failed to get container status \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": rpc error: code = NotFound desc = could not find container \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": container with ID starting with 8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87 not found: ID does not exist" Sep 09 20:01:45.159407 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159353 2577 scope.go:117] "RemoveContainer" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.159503 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159481 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:01:45.159613 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159562 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187"} err="failed to get container status \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": rpc error: code = NotFound desc = could not find container \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": container with ID starting with 02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187 not found: ID does not exist" Sep 09 20:01:45.159613 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159583 2577 scope.go:117] "RemoveContainer" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.159814 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159796 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="config-reloader" Sep 09 20:01:45.159814 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159817 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="config-reloader" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159831 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" containerName="registry" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159840 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" containerName="registry" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159852 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159857 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159866 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="prometheus" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159871 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="prometheus" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159878 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="thanos-sidecar" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159883 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="thanos-sidecar" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159812 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} err="failed to get container status \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": rpc error: code = NotFound desc = could not find container \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": container with ID starting with bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04 not found: ID does not exist" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159904 2577 scope.go:117] "RemoveContainer" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.159932 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159893 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-web" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159945 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-web" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159955 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-thanos" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159965 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-thanos" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159979 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="init-config-reloader" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.159984 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="init-config-reloader" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160052 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160060 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b35ab8c-d801-4bcf-96c7-25de0387cae1" containerName="registry" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160069 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="thanos-sidecar" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160074 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-thanos" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160082 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="config-reloader" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160091 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="kube-rbac-proxy-web" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160100 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" containerName="prometheus" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160100 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} err="failed to get container status \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": rpc error: code = NotFound desc = could not find container \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": container with ID starting with bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec not found: ID does not exist" Sep 09 20:01:45.160331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160125 2577 scope.go:117] "RemoveContainer" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.160821 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160371 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} err="failed to get container status \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": rpc error: code = NotFound desc = could not find container \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": container with ID starting with f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098 not found: ID does not exist" Sep 09 20:01:45.160821 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160389 2577 scope.go:117] "RemoveContainer" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.160821 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160639 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} err="failed to get container status \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": rpc error: code = NotFound desc = could not find container \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": container with ID starting with 08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c not found: ID does not exist" Sep 09 20:01:45.160821 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160659 2577 scope.go:117] "RemoveContainer" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.160996 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160872 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} err="failed to get container status \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": rpc error: code = NotFound desc = could not find container \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": container with ID starting with c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5 not found: ID does not exist" Sep 09 20:01:45.160996 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.160887 2577 scope.go:117] "RemoveContainer" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.161135 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161102 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} err="failed to get container status \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": rpc error: code = NotFound desc = could not find container \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": container with ID starting with 8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87 not found: ID does not exist" Sep 09 20:01:45.161178 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161137 2577 scope.go:117] "RemoveContainer" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.161404 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161385 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187"} err="failed to get container status \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": rpc error: code = NotFound desc = could not find container \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": container with ID starting with 02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187 not found: ID does not exist" Sep 09 20:01:45.161443 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161407 2577 scope.go:117] "RemoveContainer" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.161661 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161646 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} err="failed to get container status \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": rpc error: code = NotFound desc = could not find container \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": container with ID starting with bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04 not found: ID does not exist" Sep 09 20:01:45.161704 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161661 2577 scope.go:117] "RemoveContainer" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.161843 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161829 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} err="failed to get container status \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": rpc error: code = NotFound desc = could not find container \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": container with ID starting with bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec not found: ID does not exist" Sep 09 20:01:45.161886 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.161842 2577 scope.go:117] "RemoveContainer" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.162046 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162026 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} err="failed to get container status \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": rpc error: code = NotFound desc = could not find container \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": container with ID starting with f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098 not found: ID does not exist" Sep 09 20:01:45.162121 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162048 2577 scope.go:117] "RemoveContainer" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.162324 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162307 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} err="failed to get container status \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": rpc error: code = NotFound desc = could not find container \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": container with ID starting with 08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c not found: ID does not exist" Sep 09 20:01:45.162370 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162326 2577 scope.go:117] "RemoveContainer" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.162558 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162536 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} err="failed to get container status \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": rpc error: code = NotFound desc = could not find container \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": container with ID starting with c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5 not found: ID does not exist" Sep 09 20:01:45.162558 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162558 2577 scope.go:117] "RemoveContainer" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.162794 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162773 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} err="failed to get container status \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": rpc error: code = NotFound desc = could not find container \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": container with ID starting with 8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87 not found: ID does not exist" Sep 09 20:01:45.162835 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162794 2577 scope.go:117] "RemoveContainer" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.162988 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162974 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187"} err="failed to get container status \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": rpc error: code = NotFound desc = could not find container \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": container with ID starting with 02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187 not found: ID does not exist" Sep 09 20:01:45.163032 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.162989 2577 scope.go:117] "RemoveContainer" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.163175 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163160 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} err="failed to get container status \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": rpc error: code = NotFound desc = could not find container \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": container with ID starting with bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04 not found: ID does not exist" Sep 09 20:01:45.163218 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163175 2577 scope.go:117] "RemoveContainer" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.163392 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163372 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} err="failed to get container status \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": rpc error: code = NotFound desc = could not find container \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": container with ID starting with bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec not found: ID does not exist" Sep 09 20:01:45.163479 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163394 2577 scope.go:117] "RemoveContainer" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.163654 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163636 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} err="failed to get container status \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": rpc error: code = NotFound desc = could not find container \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": container with ID starting with f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098 not found: ID does not exist" Sep 09 20:01:45.163723 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163655 2577 scope.go:117] "RemoveContainer" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.163880 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163860 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} err="failed to get container status \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": rpc error: code = NotFound desc = could not find container \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": container with ID starting with 08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c not found: ID does not exist" Sep 09 20:01:45.163918 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.163881 2577 scope.go:117] "RemoveContainer" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.164121 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164104 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} err="failed to get container status \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": rpc error: code = NotFound desc = could not find container \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": container with ID starting with c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5 not found: ID does not exist" Sep 09 20:01:45.164182 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164121 2577 scope.go:117] "RemoveContainer" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.164325 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164309 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.164387 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164320 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} err="failed to get container status \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": rpc error: code = NotFound desc = could not find container \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": container with ID starting with 8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87 not found: ID does not exist" Sep 09 20:01:45.164387 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164334 2577 scope.go:117] "RemoveContainer" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.164784 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164543 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187"} err="failed to get container status \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": rpc error: code = NotFound desc = could not find container \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": container with ID starting with 02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187 not found: ID does not exist" Sep 09 20:01:45.164784 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164563 2577 scope.go:117] "RemoveContainer" containerID="bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04" Sep 09 20:01:45.164914 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164795 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04"} err="failed to get container status \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": rpc error: code = NotFound desc = could not find container \"bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04\": container with ID starting with bb5d4679d84d0a358558648cc1161d76772a181ccacc4fcb9f11faf29602ae04 not found: ID does not exist" Sep 09 20:01:45.164914 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.164816 2577 scope.go:117] "RemoveContainer" containerID="bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec" Sep 09 20:01:45.165074 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165051 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec"} err="failed to get container status \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": rpc error: code = NotFound desc = could not find container \"bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec\": container with ID starting with bafbf759e4b29d86814167b69d939927995f5ad11d67a5c8b5e03f330b84a0ec not found: ID does not exist" Sep 09 20:01:45.165145 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165074 2577 scope.go:117] "RemoveContainer" containerID="f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098" Sep 09 20:01:45.165315 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165297 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098"} err="failed to get container status \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": rpc error: code = NotFound desc = could not find container \"f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098\": container with ID starting with f7c1c798de50e2827a73a40a6fba9623df28159bae3277ce76d3ed62d4539098 not found: ID does not exist" Sep 09 20:01:45.165315 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165315 2577 scope.go:117] "RemoveContainer" containerID="08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c" Sep 09 20:01:45.165622 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165600 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c"} err="failed to get container status \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": rpc error: code = NotFound desc = could not find container \"08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c\": container with ID starting with 08bf1ede534f07f3128d2c8790d7167c34db49570920e49f27b95a9f07a5a50c not found: ID does not exist" Sep 09 20:01:45.165622 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165621 2577 scope.go:117] "RemoveContainer" containerID="c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5" Sep 09 20:01:45.165864 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165844 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5"} err="failed to get container status \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": rpc error: code = NotFound desc = could not find container \"c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5\": container with ID starting with c6c6d03ab142e795ea797b2f86ca05c151869086978b63b936b5f5098ff1bda5 not found: ID does not exist" Sep 09 20:01:45.165933 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.165871 2577 scope.go:117] "RemoveContainer" containerID="8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87" Sep 09 20:01:45.166103 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.166083 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87"} err="failed to get container status \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": rpc error: code = NotFound desc = could not find container \"8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87\": container with ID starting with 8e1a7dd1341e223b310fa75df0fef9f64dc005f3af6ab8baddc61ecdf3133f87 not found: ID does not exist" Sep 09 20:01:45.166103 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.166102 2577 scope.go:117] "RemoveContainer" containerID="02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187" Sep 09 20:01:45.166342 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.166324 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187"} err="failed to get container status \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": rpc error: code = NotFound desc = could not find container \"02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187\": container with ID starting with 02d76a20332e5d16e582d1a5d58fb8e0c7561d6684e2b0c07721431d7a4f5187 not found: ID does not exist" Sep 09 20:01:45.167433 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167418 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 20:01:45.167869 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167423 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 20:01:45.167869 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167502 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 20:01:45.167869 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167538 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 20:01:45.167869 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167676 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pw576\"" Sep 09 20:01:45.167869 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167785 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8c2llilpmuoul\"" Sep 09 20:01:45.168118 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167943 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 20:01:45.168118 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.167942 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 20:01:45.168442 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.168426 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 20:01:45.168687 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.168670 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 20:01:45.168753 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.168670 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 20:01:45.168753 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.168692 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 20:01:45.168753 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.168717 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 20:01:45.169016 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.169000 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 20:01:45.169101 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.169015 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 20:01:45.170989 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.170971 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 20:01:45.173709 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.173690 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 20:01:45.178912 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.178891 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:01:45.192530 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192609 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192541 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-config\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192609 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192564 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192609 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgr6l\" (UniqueName: \"kubernetes.io/projected/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-kube-api-access-jgr6l\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192609 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192593 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192728 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192647 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192728 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192671 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-config-out\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192728 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192687 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192728 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192728 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192908 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192745 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192908 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192908 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192908 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192908 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.192908 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.193153 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192936 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.193153 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.192968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.193153 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.193004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.193153 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.193030 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-web-config\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.293658 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293631 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293680 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293769 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293838 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.293876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.294641 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294590 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-web-config\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-config\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294743 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294798 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgr6l\" (UniqueName: \"kubernetes.io/projected/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-kube-api-access-jgr6l\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294822 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-config-out\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295062 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.294932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295660 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.295077 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.295719 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.295666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.297645 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.297574 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.297771 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.297738 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.299811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.298242 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.299811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.298647 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.299811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.298706 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.299811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.299136 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.299811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.299323 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.299811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.299480 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-config\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.299811 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.299710 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.300173 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.300014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.300173 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.300127 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.304809 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.304787 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-config-out\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.304984 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.304961 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.304984 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.304975 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.306151 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.306129 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-web-config\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.306309 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.306290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgr6l\" (UniqueName: \"kubernetes.io/projected/8ae542f2-22d5-4e61-a658-a1aa6f0d9f88-kube-api-access-jgr6l\") pod \"prometheus-k8s-0\" (UID: \"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.475054 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.475025 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:01:45.598286 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.598256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 20:01:45.600558 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.600540 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379c5732-e859-4f78-895c-62bd2f25915b-metrics-certs\") pod \"network-metrics-daemon-98kdp\" (UID: \"379c5732-e859-4f78-895c-62bd2f25915b\") " pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 20:01:45.756087 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.756061 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:01:45.756523 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:01:45.756503 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae542f2_22d5_4e61_a658_a1aa6f0d9f88.slice/crio-b3a3bb8bfc931257a2a935ad59cd7e89165b34d21885d1af5c1bb7155fb33540 WatchSource:0}: Error finding container b3a3bb8bfc931257a2a935ad59cd7e89165b34d21885d1af5c1bb7155fb33540: Status 404 returned error can't find the container with id b3a3bb8bfc931257a2a935ad59cd7e89165b34d21885d1af5c1bb7155fb33540 Sep 09 20:01:45.779543 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.779522 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cpmqx\"" Sep 09 20:01:45.786898 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.786883 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98kdp" Sep 09 20:01:45.880424 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:45.880396 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601" path="/var/lib/kubelet/pods/6fa1e5d4-ae78-4ef5-a3e3-0eaee4855601/volumes" Sep 09 20:01:46.059295 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:46.059273 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98kdp"] Sep 09 20:01:46.061678 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:01:46.061656 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379c5732_e859_4f78_895c_62bd2f25915b.slice/crio-51e9dbbc8662022cd4ff485dc71005ef1ba1a41755ef22b0b12d87390c7e811b WatchSource:0}: Error finding container 51e9dbbc8662022cd4ff485dc71005ef1ba1a41755ef22b0b12d87390c7e811b: Status 404 returned error can't find the container with id 51e9dbbc8662022cd4ff485dc71005ef1ba1a41755ef22b0b12d87390c7e811b Sep 09 20:01:46.063511 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:46.063498 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 20:01:46.108844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:46.108821 2577 generic.go:356] "Generic (PLEG): container finished" podID="8ae542f2-22d5-4e61-a658-a1aa6f0d9f88" containerID="137bc8ae08d71d39f286915275d4d8e899cced14e3077318b3a91872ed734da8" exitCode=0 Sep 09 20:01:46.108968 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:46.108912 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerDied","Data":"137bc8ae08d71d39f286915275d4d8e899cced14e3077318b3a91872ed734da8"} Sep 09 20:01:46.109027 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:46.108988 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerStarted","Data":"b3a3bb8bfc931257a2a935ad59cd7e89165b34d21885d1af5c1bb7155fb33540"} Sep 09 20:01:46.110175 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:46.110155 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98kdp" event={"ID":"379c5732-e859-4f78-895c-62bd2f25915b","Type":"ContainerStarted","Data":"51e9dbbc8662022cd4ff485dc71005ef1ba1a41755ef22b0b12d87390c7e811b"} Sep 09 20:01:47.118342 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:47.118315 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerStarted","Data":"64dd812e07a30cec48c4bb7b5f8ebb4310e0be8da828ede4507d371aa3661204"} Sep 09 20:01:47.118664 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:47.118390 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerStarted","Data":"78e5fee3385900581f054d56c99d07e964f2e17509b16b4d947c53f56d832f12"} Sep 09 20:01:47.118664 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:47.118432 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerStarted","Data":"ca6e5f5e4f6b2c76b8ebc36c786a2a6f562e3d56f9ced440b3579dc26e800537"} Sep 09 20:01:47.118664 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:47.118493 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerStarted","Data":"23f00ca5d06a17f33675b3ed605561e7c564650091dfcd40583da4b234eb7836"} Sep 09 20:01:47.118664 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:47.118519 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerStarted","Data":"f4d971e50935654ef812beca5d9309c23cf336001f1369014467f28691fff65c"} Sep 09 20:01:47.118664 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:47.118559 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8ae542f2-22d5-4e61-a658-a1aa6f0d9f88","Type":"ContainerStarted","Data":"54c808ecd1a70865a82771d01a5c69d45cd4dc5ba8d5cc43129ede97205d279d"} Sep 09 20:01:47.151920 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:47.151503 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.12804539 podStartE2EDuration="2.151482531s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.437141ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:01:47.148734977 +0000 UTC m=+375.773239014" watchObservedRunningTime="2026-09-09 20:01:47.151482531 +0000 UTC m=+375.775986568" Sep 09 20:01:48.123523 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:48.123485 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98kdp" event={"ID":"379c5732-e859-4f78-895c-62bd2f25915b","Type":"ContainerStarted","Data":"c7b3ab06ff47cb6df84e53b3a41144bda07ad170acc4b54f4ffbd2237c2e6419"} Sep 09 20:01:48.123887 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:48.123540 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98kdp" event={"ID":"379c5732-e859-4f78-895c-62bd2f25915b","Type":"ContainerStarted","Data":"c2d7a801428693eaff8e0ab91a48a3c699a96ed4b4d0d9c9405a13d02b9427b8"} Sep 09 20:01:48.143602 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:48.143554 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-98kdp" podStartSLOduration=376.098216874 podStartE2EDuration="6m17.143541162s" totalImagesPullingTime="1.045324288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 19:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:01:48.142949786 +0000 UTC m=+376.767453822" watchObservedRunningTime="2026-09-09 20:01:48.143541162 +0000 UTC m=+376.768045198" Sep 09 20:01:50.476100 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:01:50.476044 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:02:45.475536 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:02:45.475491 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:02:45.491209 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:02:45.491186 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:02:46.366638 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:02:46.366612 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:05:31.796061 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:05:31.796036 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:05:31.796061 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:05:31.796048 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:10:18.794112 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.794083 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-v7h2c"] Sep 09 20:10:18.797531 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.797510 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-v7h2c" Sep 09 20:10:18.800530 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.800510 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\"" Sep 09 20:10:18.800740 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.800720 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Sep 09 20:10:18.800790 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.800728 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-v5gh2\"" Sep 09 20:10:18.802018 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.802001 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Sep 09 20:10:18.802138 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.802089 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Sep 09 20:10:18.807794 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.807771 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-v7h2c"] Sep 09 20:10:18.844377 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.844354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmvws\" (UniqueName: \"kubernetes.io/projected/c0d3cb33-b938-466c-82d8-acfbae001265-kube-api-access-bmvws\") pod \"s3-init-v7h2c\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " pod="kserve/s3-init-v7h2c" Sep 09 20:10:18.844508 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.844428 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0d3cb33-b938-466c-82d8-acfbae001265-model-cache\") pod \"s3-init-v7h2c\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " pod="kserve/s3-init-v7h2c" Sep 09 20:10:18.945391 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.945366 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmvws\" (UniqueName: \"kubernetes.io/projected/c0d3cb33-b938-466c-82d8-acfbae001265-kube-api-access-bmvws\") pod \"s3-init-v7h2c\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " pod="kserve/s3-init-v7h2c" Sep 09 20:10:18.945503 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.945413 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0d3cb33-b938-466c-82d8-acfbae001265-model-cache\") pod \"s3-init-v7h2c\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " pod="kserve/s3-init-v7h2c" Sep 09 20:10:18.945736 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.945721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0d3cb33-b938-466c-82d8-acfbae001265-model-cache\") pod \"s3-init-v7h2c\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " pod="kserve/s3-init-v7h2c" Sep 09 20:10:18.954696 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:18.954679 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmvws\" (UniqueName: \"kubernetes.io/projected/c0d3cb33-b938-466c-82d8-acfbae001265-kube-api-access-bmvws\") pod \"s3-init-v7h2c\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " pod="kserve/s3-init-v7h2c" Sep 09 20:10:19.138937 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:19.138830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-v7h2c" Sep 09 20:10:19.410870 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:19.410845 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-v7h2c"] Sep 09 20:10:19.413269 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:10:19.413241 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0d3cb33_b938_466c_82d8_acfbae001265.slice/crio-1704d3393885ae53e0f5511c9554a35248e44f91523aae95fd17b7c9a9f5fe63 WatchSource:0}: Error finding container 1704d3393885ae53e0f5511c9554a35248e44f91523aae95fd17b7c9a9f5fe63: Status 404 returned error can't find the container with id 1704d3393885ae53e0f5511c9554a35248e44f91523aae95fd17b7c9a9f5fe63 Sep 09 20:10:19.415058 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:19.415043 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 20:10:20.024790 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:20.024760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-v7h2c" event={"ID":"c0d3cb33-b938-466c-82d8-acfbae001265","Type":"ContainerStarted","Data":"1704d3393885ae53e0f5511c9554a35248e44f91523aae95fd17b7c9a9f5fe63"} Sep 09 20:10:24.043810 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:24.043771 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-v7h2c" event={"ID":"c0d3cb33-b938-466c-82d8-acfbae001265","Type":"ContainerStarted","Data":"be6886b7f5d1771f37e5c4013edee1aba80ee55ac2364d68a37ce83c08d1de8e"} Sep 09 20:10:25.049610 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:25.049575 2577 generic.go:356] "Generic (PLEG): container finished" podID="c0d3cb33-b938-466c-82d8-acfbae001265" containerID="be6886b7f5d1771f37e5c4013edee1aba80ee55ac2364d68a37ce83c08d1de8e" exitCode=0 Sep 09 20:10:25.049610 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:25.049614 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-v7h2c" event={"ID":"c0d3cb33-b938-466c-82d8-acfbae001265","Type":"ContainerDied","Data":"be6886b7f5d1771f37e5c4013edee1aba80ee55ac2364d68a37ce83c08d1de8e"} Sep 09 20:10:30.073313 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:30.073232 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-v7h2c" event={"ID":"c0d3cb33-b938-466c-82d8-acfbae001265","Type":"ContainerStarted","Data":"316c47dfe04edea24de1c9c6be0ca9ecc8bf050fa07bb0ccc9e3ad7d48b89b90"} Sep 09 20:10:30.094106 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:30.094065 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-v7h2c" podStartSLOduration=1.9367969619999998 podStartE2EDuration="12.094052359s" totalImagesPullingTime="8.482875506s" totalInitContainerRuntime="1.674379891s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:10:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:10:30.092628782 +0000 UTC m=+898.717132819" watchObservedRunningTime="2026-09-09 20:10:30.094052359 +0000 UTC m=+898.718556395" Sep 09 20:10:31.825321 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:31.825228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:10:31.825800 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:10:31.825564 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:11:35.321836 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:35.321795 2577 generic.go:356] "Generic (PLEG): container finished" podID="c0d3cb33-b938-466c-82d8-acfbae001265" containerID="316c47dfe04edea24de1c9c6be0ca9ecc8bf050fa07bb0ccc9e3ad7d48b89b90" exitCode=0 Sep 09 20:11:35.322230 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:35.321870 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-v7h2c" event={"ID":"c0d3cb33-b938-466c-82d8-acfbae001265","Type":"ContainerDied","Data":"316c47dfe04edea24de1c9c6be0ca9ecc8bf050fa07bb0ccc9e3ad7d48b89b90"} Sep 09 20:11:36.447524 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:36.447500 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-v7h2c" Sep 09 20:11:36.587648 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:36.587576 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0d3cb33-b938-466c-82d8-acfbae001265-model-cache\") pod \"c0d3cb33-b938-466c-82d8-acfbae001265\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " Sep 09 20:11:36.587783 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:36.587663 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmvws\" (UniqueName: \"kubernetes.io/projected/c0d3cb33-b938-466c-82d8-acfbae001265-kube-api-access-bmvws\") pod \"c0d3cb33-b938-466c-82d8-acfbae001265\" (UID: \"c0d3cb33-b938-466c-82d8-acfbae001265\") " Sep 09 20:11:36.589955 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:36.589930 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d3cb33-b938-466c-82d8-acfbae001265-kube-api-access-bmvws" (OuterVolumeSpecName: "kube-api-access-bmvws") pod "c0d3cb33-b938-466c-82d8-acfbae001265" (UID: "c0d3cb33-b938-466c-82d8-acfbae001265"). InnerVolumeSpecName "kube-api-access-bmvws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:11:36.609625 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:36.609598 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d3cb33-b938-466c-82d8-acfbae001265-model-cache" (OuterVolumeSpecName: "model-cache") pod "c0d3cb33-b938-466c-82d8-acfbae001265" (UID: "c0d3cb33-b938-466c-82d8-acfbae001265"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:11:36.688293 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:36.688268 2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0d3cb33-b938-466c-82d8-acfbae001265-model-cache\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:11:36.688293 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:36.688292 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmvws\" (UniqueName: \"kubernetes.io/projected/c0d3cb33-b938-466c-82d8-acfbae001265-kube-api-access-bmvws\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:11:37.330844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:37.330814 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-v7h2c" Sep 09 20:11:37.331005 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:37.330848 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-v7h2c" event={"ID":"c0d3cb33-b938-466c-82d8-acfbae001265","Type":"ContainerDied","Data":"1704d3393885ae53e0f5511c9554a35248e44f91523aae95fd17b7c9a9f5fe63"} Sep 09 20:11:37.331005 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:37.330885 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1704d3393885ae53e0f5511c9554a35248e44f91523aae95fd17b7c9a9f5fe63" Sep 09 20:11:50.392848 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.392815 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f"] Sep 09 20:11:50.393206 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.393108 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d3cb33-b938-466c-82d8-acfbae001265" containerName="s3-init" Sep 09 20:11:50.393206 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.393119 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d3cb33-b938-466c-82d8-acfbae001265" containerName="s3-init" Sep 09 20:11:50.393206 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.393131 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d3cb33-b938-466c-82d8-acfbae001265" containerName="download-hf-model" Sep 09 20:11:50.393206 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.393137 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d3cb33-b938-466c-82d8-acfbae001265" containerName="download-hf-model" Sep 09 20:11:50.393206 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.393189 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d3cb33-b938-466c-82d8-acfbae001265" containerName="s3-init" Sep 09 20:11:50.396564 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.396547 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.399372 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.399350 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 09 20:11:50.399514 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.399356 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 09 20:11:50.399679 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.399665 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 09 20:11:50.400744 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.400718 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w6n8h\"" Sep 09 20:11:50.400744 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.400735 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Sep 09 20:11:50.400913 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.400738 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 09 20:11:50.400913 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.400846 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Sep 09 20:11:50.402221 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.402201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.402307 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.402237 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-485wz\" (UniqueName: \"kubernetes.io/projected/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-api-access-485wz\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.402307 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.402263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.402428 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.402321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d9effee7-acc2-4a07-aeb8-1188b17afea3-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.402504 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.402423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.405535 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.405513 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f"] Sep 09 20:11:50.502766 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.502737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-485wz\" (UniqueName: \"kubernetes.io/projected/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-api-access-485wz\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.502947 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.502777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.502947 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.502803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d9effee7-acc2-4a07-aeb8-1188b17afea3-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.502947 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.502862 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.502947 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.502898 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.502947 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:11:50.502908 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Sep 09 20:11:50.503202 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:11:50.502976 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls podName:d9effee7-acc2-4a07-aeb8-1188b17afea3 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:51.002956491 +0000 UTC m=+979.627460507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls") pod "isvc-sklearn-graph-1-predictor-675c9cf848-w966f" (UID: "d9effee7-acc2-4a07-aeb8-1188b17afea3") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Sep 09 20:11:50.503269 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.503235 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d9effee7-acc2-4a07-aeb8-1188b17afea3-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.503684 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.503661 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.503804 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.503790 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.517480 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.517372 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-485wz\" (UniqueName: \"kubernetes.io/projected/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-api-access-485wz\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:50.891329 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.891294 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b"] Sep 09 20:11:50.894868 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.894846 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:50.897630 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.897612 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Sep 09 20:11:50.897944 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.897926 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Sep 09 20:11:50.905667 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.905643 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:50.905834 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.905817 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:50.905989 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.905966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:50.906088 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.906077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96mdd\" (UniqueName: \"kubernetes.io/projected/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-api-access-96mdd\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:50.906169 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.906130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:50.908519 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:50.908497 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b"] Sep 09 20:11:51.006952 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.006923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:51.007144 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.006962 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.007144 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.006982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.007144 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.007009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.007144 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.007037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96mdd\" (UniqueName: \"kubernetes.io/projected/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-api-access-96mdd\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.007144 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.007056 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.007144 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:11:51.007079 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Sep 09 20:11:51.007144 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:11:51.007147 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls podName:d9effee7-acc2-4a07-aeb8-1188b17afea3 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:52.007127475 +0000 UTC m=+980.631631497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls") pod "isvc-sklearn-graph-1-predictor-675c9cf848-w966f" (UID: "d9effee7-acc2-4a07-aeb8-1188b17afea3") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Sep 09 20:11:51.007548 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:11:51.007189 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Sep 09 20:11:51.007548 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:11:51.007258 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls podName:22da0421-7cee-4178-b8bc-0604cf2f3ba4 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:51.507235755 +0000 UTC m=+980.131739770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls") pod "isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" (UID: "22da0421-7cee-4178-b8bc-0604cf2f3ba4") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Sep 09 20:11:51.007548 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.007397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.007750 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.007732 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.007750 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.007744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.016954 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.016927 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96mdd\" (UniqueName: \"kubernetes.io/projected/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-api-access-96mdd\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.510408 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.510378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.512740 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.512715 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-667cb66696-ddx2b\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:51.806417 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:51.806337 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:11:52.013975 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:52.013948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:52.016437 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:52.016412 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-675c9cf848-w966f\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:52.079610 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:52.079544 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b"] Sep 09 20:11:52.083007 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:11:52.082972 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22da0421_7cee_4178_b8bc_0604cf2f3ba4.slice/crio-0db073cce2ade641ea1010f2701b7416df734b5e3bddb3631ffdd7f84fbea89d WatchSource:0}: Error finding container 0db073cce2ade641ea1010f2701b7416df734b5e3bddb3631ffdd7f84fbea89d: Status 404 returned error can't find the container with id 0db073cce2ade641ea1010f2701b7416df734b5e3bddb3631ffdd7f84fbea89d Sep 09 20:11:52.209846 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:52.209812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:11:52.389920 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:52.389844 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerStarted","Data":"ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee"} Sep 09 20:11:52.389920 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:52.389903 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerStarted","Data":"0db073cce2ade641ea1010f2701b7416df734b5e3bddb3631ffdd7f84fbea89d"} Sep 09 20:11:52.483471 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:52.483430 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f"] Sep 09 20:11:52.486056 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:11:52.486028 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9effee7_acc2_4a07_aeb8_1188b17afea3.slice/crio-d7d2fedb3fd6691ca1c4966baf243b9003ef4f5fd56c0cd95d6520447bbfc536 WatchSource:0}: Error finding container d7d2fedb3fd6691ca1c4966baf243b9003ef4f5fd56c0cd95d6520447bbfc536: Status 404 returned error can't find the container with id d7d2fedb3fd6691ca1c4966baf243b9003ef4f5fd56c0cd95d6520447bbfc536 Sep 09 20:11:53.395759 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:53.395669 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerStarted","Data":"3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c"} Sep 09 20:11:53.395759 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:53.395728 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerStarted","Data":"d7d2fedb3fd6691ca1c4966baf243b9003ef4f5fd56c0cd95d6520447bbfc536"} Sep 09 20:11:56.407446 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:56.407415 2577 generic.go:356] "Generic (PLEG): container finished" podID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerID="ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee" exitCode=0 Sep 09 20:11:56.407858 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:56.407492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerDied","Data":"ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee"} Sep 09 20:11:56.409170 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:56.409092 2577 generic.go:356] "Generic (PLEG): container finished" podID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerID="3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c" exitCode=0 Sep 09 20:11:56.409170 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:11:56.409150 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerDied","Data":"3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c"} Sep 09 20:12:14.549605 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:14.549473 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerStarted","Data":"6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5"} Sep 09 20:12:14.552882 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:14.552847 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerStarted","Data":"1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24"} Sep 09 20:12:16.564174 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.564078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerStarted","Data":"e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac"} Sep 09 20:12:16.564638 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.564282 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:12:16.564638 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.564440 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:12:16.566000 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.565969 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:12:16.566604 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.566584 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerStarted","Data":"3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e"} Sep 09 20:12:16.566733 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.566717 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:12:16.566801 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.566769 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:12:16.567823 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.567803 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:12:16.584742 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.584702 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podStartSLOduration=3.020133178 podStartE2EDuration="26.584690357s" totalImagesPullingTime="19.803745734s" totalInitContainerRuntime="3.760811445s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:11:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:16.583620838 +0000 UTC m=+1005.208124875" watchObservedRunningTime="2026-09-09 20:12:16.584690357 +0000 UTC m=+1005.209194394" Sep 09 20:12:16.605087 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:16.605033 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podStartSLOduration=2.983559415 podStartE2EDuration="26.605014811s" totalImagesPullingTime="19.793696022s" totalInitContainerRuntime="3.827759374s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:11:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:16.603202445 +0000 UTC m=+1005.227706483" watchObservedRunningTime="2026-09-09 20:12:16.605014811 +0000 UTC m=+1005.229518850" Sep 09 20:12:17.571851 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:17.571806 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:12:17.572261 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:17.571933 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:12:22.578261 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:22.578230 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:12:22.578844 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:22.578814 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:12:22.578985 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:22.578965 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:12:22.579435 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:22.579413 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:12:32.579639 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:32.579603 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:12:32.580091 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:32.579606 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:12:42.578688 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:42.578652 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:12:42.579441 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:42.579420 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:12:52.579564 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:52.579519 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:12:52.580040 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:12:52.579520 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:13:02.578833 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:02.578792 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:13:02.580026 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:02.580003 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:13:12.579330 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:12.579287 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:13:12.579806 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:12.579422 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:13:22.579220 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:22.579174 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:13:22.579631 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:22.579404 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 09 20:13:29.879710 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:29.879683 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:13:32.579764 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:13:32.579738 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:14:10.522894 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:10.522864 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b"] Sep 09 20:14:10.523407 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:10.523102 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" containerID="cri-o://6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5" gracePeriod=30 Sep 09 20:14:10.523407 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:10.523157 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kube-rbac-proxy" containerID="cri-o://3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e" gracePeriod=30 Sep 09 20:14:10.557307 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:10.557278 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f"] Sep 09 20:14:10.557568 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:10.557547 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" containerID="cri-o://1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24" gracePeriod=30 Sep 09 20:14:10.557650 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:10.557581 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kube-rbac-proxy" containerID="cri-o://e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac" gracePeriod=30 Sep 09 20:14:11.037294 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:11.037258 2577 generic.go:356] "Generic (PLEG): container finished" podID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerID="e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac" exitCode=2 Sep 09 20:14:11.037494 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:11.037333 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerDied","Data":"e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac"} Sep 09 20:14:11.039591 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:11.039568 2577 generic.go:356] "Generic (PLEG): container finished" podID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerID="3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e" exitCode=2 Sep 09 20:14:11.039695 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:11.039598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerDied","Data":"3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e"} Sep 09 20:14:12.571995 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:12.571951 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": dial tcp 10.134.0.21:8643: connect: connection refused" Sep 09 20:14:12.572392 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:12.571951 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.22:8643/healthz\": dial tcp 10.134.0.22:8643: connect: connection refused" Sep 09 20:14:12.579482 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:12.579436 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 09 20:14:15.386083 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.386060 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:14:15.414488 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.414437 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:14:15.520599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520520 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-cabundle-cert\") pod \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " Sep 09 20:14:15.520599 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520577 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-rbac-proxy-sar-config\") pod \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " Sep 09 20:14:15.520819 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520610 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-cabundle-cert\") pod \"d9effee7-acc2-4a07-aeb8-1188b17afea3\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " Sep 09 20:14:15.520819 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520636 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-rbac-proxy-sar-config\") pod \"d9effee7-acc2-4a07-aeb8-1188b17afea3\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " Sep 09 20:14:15.520819 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520682 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96mdd\" (UniqueName: \"kubernetes.io/projected/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-api-access-96mdd\") pod \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " Sep 09 20:14:15.520973 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520884 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-485wz\" (UniqueName: \"kubernetes.io/projected/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-api-access-485wz\") pod \"d9effee7-acc2-4a07-aeb8-1188b17afea3\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " Sep 09 20:14:15.520973 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520926 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kserve-provision-location\") pod \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " Sep 09 20:14:15.520973 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520954 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d9effee7-acc2-4a07-aeb8-1188b17afea3-kserve-provision-location\") pod \"d9effee7-acc2-4a07-aeb8-1188b17afea3\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " Sep 09 20:14:15.521136 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520969 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "22da0421-7cee-4178-b8bc-0604cf2f3ba4" (UID: "22da0421-7cee-4178-b8bc-0604cf2f3ba4"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:14:15.521136 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520982 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls\") pod \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\" (UID: \"22da0421-7cee-4178-b8bc-0604cf2f3ba4\") " Sep 09 20:14:15.521136 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.520994 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "22da0421-7cee-4178-b8bc-0604cf2f3ba4" (UID: "22da0421-7cee-4178-b8bc-0604cf2f3ba4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:14:15.521136 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521033 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d9effee7-acc2-4a07-aeb8-1188b17afea3" (UID: "d9effee7-acc2-4a07-aeb8-1188b17afea3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:14:15.521136 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521038 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d9effee7-acc2-4a07-aeb8-1188b17afea3" (UID: "d9effee7-acc2-4a07-aeb8-1188b17afea3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:14:15.521350 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521246 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls\") pod \"d9effee7-acc2-4a07-aeb8-1188b17afea3\" (UID: \"d9effee7-acc2-4a07-aeb8-1188b17afea3\") " Sep 09 20:14:15.521350 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521246 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "22da0421-7cee-4178-b8bc-0604cf2f3ba4" (UID: "22da0421-7cee-4178-b8bc-0604cf2f3ba4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:14:15.521434 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521354 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9effee7-acc2-4a07-aeb8-1188b17afea3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d9effee7-acc2-4a07-aeb8-1188b17afea3" (UID: "d9effee7-acc2-4a07-aeb8-1188b17afea3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:14:15.521561 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521540 2577 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-cabundle-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.521561 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521562 2577 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.521684 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521571 2577 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-cabundle-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.521684 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521579 2577 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.521684 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521587 2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kserve-provision-location\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.521684 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.521596 2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d9effee7-acc2-4a07-aeb8-1188b17afea3-kserve-provision-location\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.523115 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.523092 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-api-access-96mdd" (OuterVolumeSpecName: "kube-api-access-96mdd") pod "22da0421-7cee-4178-b8bc-0604cf2f3ba4" (UID: "22da0421-7cee-4178-b8bc-0604cf2f3ba4"). InnerVolumeSpecName "kube-api-access-96mdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:14:15.523297 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.523265 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-api-access-485wz" (OuterVolumeSpecName: "kube-api-access-485wz") pod "d9effee7-acc2-4a07-aeb8-1188b17afea3" (UID: "d9effee7-acc2-4a07-aeb8-1188b17afea3"). InnerVolumeSpecName "kube-api-access-485wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:14:15.523672 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.523657 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "22da0421-7cee-4178-b8bc-0604cf2f3ba4" (UID: "22da0421-7cee-4178-b8bc-0604cf2f3ba4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:14:15.523717 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.523689 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d9effee7-acc2-4a07-aeb8-1188b17afea3" (UID: "d9effee7-acc2-4a07-aeb8-1188b17afea3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:14:15.622783 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.622744 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96mdd\" (UniqueName: \"kubernetes.io/projected/22da0421-7cee-4178-b8bc-0604cf2f3ba4-kube-api-access-96mdd\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.622783 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.622779 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-485wz\" (UniqueName: \"kubernetes.io/projected/d9effee7-acc2-4a07-aeb8-1188b17afea3-kube-api-access-485wz\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.622942 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.622792 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22da0421-7cee-4178-b8bc-0604cf2f3ba4-proxy-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:15.622942 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:15.622806 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d9effee7-acc2-4a07-aeb8-1188b17afea3-proxy-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:14:16.060417 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.060381 2577 generic.go:356] "Generic (PLEG): container finished" podID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerID="6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5" exitCode=0 Sep 09 20:14:16.060589 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.060426 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerDied","Data":"6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5"} Sep 09 20:14:16.060589 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.060486 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" Sep 09 20:14:16.060589 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.060506 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b" event={"ID":"22da0421-7cee-4178-b8bc-0604cf2f3ba4","Type":"ContainerDied","Data":"0db073cce2ade641ea1010f2701b7416df734b5e3bddb3631ffdd7f84fbea89d"} Sep 09 20:14:16.060589 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.060533 2577 scope.go:117] "RemoveContainer" containerID="3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e" Sep 09 20:14:16.062770 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.062741 2577 generic.go:356] "Generic (PLEG): container finished" podID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerID="1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24" exitCode=0 Sep 09 20:14:16.062902 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.062799 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerDied","Data":"1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24"} Sep 09 20:14:16.062902 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.062811 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" Sep 09 20:14:16.062902 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.062859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f" event={"ID":"d9effee7-acc2-4a07-aeb8-1188b17afea3","Type":"ContainerDied","Data":"d7d2fedb3fd6691ca1c4966baf243b9003ef4f5fd56c0cd95d6520447bbfc536"} Sep 09 20:14:16.069327 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.069312 2577 scope.go:117] "RemoveContainer" containerID="6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5" Sep 09 20:14:16.076555 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.076539 2577 scope.go:117] "RemoveContainer" containerID="ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee" Sep 09 20:14:16.082034 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.082013 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f"] Sep 09 20:14:16.084098 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.084084 2577 scope.go:117] "RemoveContainer" containerID="3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e" Sep 09 20:14:16.084350 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:14:16.084334 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e\": container with ID starting with 3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e not found: ID does not exist" containerID="3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e" Sep 09 20:14:16.084422 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.084361 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e"} err="failed to get container status \"3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e\": rpc error: code = NotFound desc = could not find container \"3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e\": container with ID starting with 3375336b4c49d3651c635cc91d4ee387cca6fa1659f1bfb62f95c105798aa95e not found: ID does not exist" Sep 09 20:14:16.084422 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.084385 2577 scope.go:117] "RemoveContainer" containerID="6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5" Sep 09 20:14:16.084689 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:14:16.084673 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5\": container with ID starting with 6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5 not found: ID does not exist" containerID="6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5" Sep 09 20:14:16.084729 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.084695 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5"} err="failed to get container status \"6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5\": rpc error: code = NotFound desc = could not find container \"6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5\": container with ID starting with 6cb1c2dd8bd7bbc6889f861cb0a84637304ad1c55681bed3454f1bccd567b7e5 not found: ID does not exist" Sep 09 20:14:16.084729 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.084714 2577 scope.go:117] "RemoveContainer" containerID="ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee" Sep 09 20:14:16.084940 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:14:16.084924 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee\": container with ID starting with ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee not found: ID does not exist" containerID="ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee" Sep 09 20:14:16.084987 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.084946 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee"} err="failed to get container status \"ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee\": rpc error: code = NotFound desc = could not find container \"ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee\": container with ID starting with ea1802d1f3b910afbb9f19439f36d52ebee186597073294e712674e1b153fcee not found: ID does not exist" Sep 09 20:14:16.084987 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.084961 2577 scope.go:117] "RemoveContainer" containerID="e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac" Sep 09 20:14:16.089674 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.089655 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-675c9cf848-w966f"] Sep 09 20:14:16.092331 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.092313 2577 scope.go:117] "RemoveContainer" containerID="1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24" Sep 09 20:14:16.101260 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.101245 2577 scope.go:117] "RemoveContainer" containerID="3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c" Sep 09 20:14:16.103575 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.103559 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b"] Sep 09 20:14:16.107676 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.107656 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-667cb66696-ddx2b"] Sep 09 20:14:16.109399 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.109383 2577 scope.go:117] "RemoveContainer" containerID="e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac" Sep 09 20:14:16.109714 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:14:16.109698 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac\": container with ID starting with e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac not found: ID does not exist" containerID="e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac" Sep 09 20:14:16.109765 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.109721 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac"} err="failed to get container status \"e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac\": rpc error: code = NotFound desc = could not find container \"e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac\": container with ID starting with e9bcabdb1bdb91b7777a1e5aa916f7ee23f72821ee84f3a1e004c5b043cf95ac not found: ID does not exist" Sep 09 20:14:16.109765 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.109737 2577 scope.go:117] "RemoveContainer" containerID="1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24" Sep 09 20:14:16.110020 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:14:16.110003 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24\": container with ID starting with 1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24 not found: ID does not exist" containerID="1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24" Sep 09 20:14:16.110087 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.110028 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24"} err="failed to get container status \"1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24\": rpc error: code = NotFound desc = could not find container \"1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24\": container with ID starting with 1f2ddd67c6cdffe05b246a6b776b0265d0ed9265363a53d0c011035b1e641f24 not found: ID does not exist" Sep 09 20:14:16.110087 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.110051 2577 scope.go:117] "RemoveContainer" containerID="3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c" Sep 09 20:14:16.110334 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:14:16.110311 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c\": container with ID starting with 3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c not found: ID does not exist" containerID="3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c" Sep 09 20:14:16.110428 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:16.110340 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c"} err="failed to get container status \"3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c\": rpc error: code = NotFound desc = could not find container \"3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c\": container with ID starting with 3d52b66ad76baf180d6bf02cb1f049dbc9931a4aa7c276a40302635926498a8c not found: ID does not exist" Sep 09 20:14:17.879782 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:17.879750 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" path="/var/lib/kubelet/pods/22da0421-7cee-4178-b8bc-0604cf2f3ba4/volumes" Sep 09 20:14:17.880231 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:14:17.880217 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" path="/var/lib/kubelet/pods/d9effee7-acc2-4a07-aeb8-1188b17afea3/volumes" Sep 09 20:15:31.853214 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:15:31.853185 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:15:31.853804 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:15:31.853663 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:20:31.878873 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:20:31.878847 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:20:31.880042 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:20:31.880015 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:25:31.904566 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:25:31.904540 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:25:31.906023 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:25:31.906003 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:30:31.931543 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:30:31.931518 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:30:31.934708 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:30:31.934425 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:35:31.956031 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:35:31.955920 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:35:31.960267 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:35:31.960243 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:40:31.981836 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:40:31.981722 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:40:31.986654 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:40:31.986632 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:45:32.006951 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:45:32.006840 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:45:32.013247 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:45:32.013221 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:50:32.034140 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:50:32.034036 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:50:32.043189 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:50:32.043165 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:51:30.992270 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992237 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tjbpx/must-gather-hw4jr"] Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992528 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kube-rbac-proxy" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992540 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kube-rbac-proxy" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992555 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="storage-initializer" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992562 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="storage-initializer" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992571 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="storage-initializer" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992576 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="storage-initializer" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992583 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kube-rbac-proxy" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992588 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kube-rbac-proxy" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992602 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992607 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992614 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992619 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992662 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kserve-container" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992672 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kserve-container" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992678 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="22da0421-7cee-4178-b8bc-0604cf2f3ba4" containerName="kube-rbac-proxy" Sep 09 20:51:30.992760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.992684 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9effee7-acc2-4a07-aeb8-1188b17afea3" containerName="kube-rbac-proxy" Sep 09 20:51:30.995707 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.995691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:30.998760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.998733 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tjbpx\"/\"kube-root-ca.crt\"" Sep 09 20:51:30.998877 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:30.998741 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tjbpx\"/\"openshift-service-ca.crt\"" Sep 09 20:51:31.000059 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.000028 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tjbpx\"/\"default-dockercfg-wj6r7\"" Sep 09 20:51:31.004010 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.003989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjbpx/must-gather-hw4jr"] Sep 09 20:51:31.056521 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.056499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcl4x\" (UniqueName: \"kubernetes.io/projected/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-kube-api-access-xcl4x\") pod \"must-gather-hw4jr\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:31.056648 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.056568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-must-gather-output\") pod \"must-gather-hw4jr\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:31.157837 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.157797 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-must-gather-output\") pod \"must-gather-hw4jr\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:31.157984 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.157851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcl4x\" (UniqueName: \"kubernetes.io/projected/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-kube-api-access-xcl4x\") pod \"must-gather-hw4jr\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:31.158119 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.158100 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-must-gather-output\") pod \"must-gather-hw4jr\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:31.169809 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.169776 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcl4x\" (UniqueName: \"kubernetes.io/projected/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-kube-api-access-xcl4x\") pod \"must-gather-hw4jr\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:31.311791 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.311708 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:51:31.581162 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.581043 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tjbpx/must-gather-hw4jr"] Sep 09 20:51:31.583745 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:51:31.583718 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d24b9db_2715_4cd4_9cb2_fbd26f68dd91.slice/crio-38535cfa44c00ddf3c102cff87903d4140f27cb964d299a0752d0014cdf1626b WatchSource:0}: Error finding container 38535cfa44c00ddf3c102cff87903d4140f27cb964d299a0752d0014cdf1626b: Status 404 returned error can't find the container with id 38535cfa44c00ddf3c102cff87903d4140f27cb964d299a0752d0014cdf1626b Sep 09 20:51:31.585547 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:31.585532 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 20:51:32.516380 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:32.516333 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" event={"ID":"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91","Type":"ContainerStarted","Data":"38535cfa44c00ddf3c102cff87903d4140f27cb964d299a0752d0014cdf1626b"} Sep 09 20:51:38.547563 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:38.547526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" event={"ID":"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91","Type":"ContainerStarted","Data":"b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd"} Sep 09 20:51:38.547563 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:38.547568 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" event={"ID":"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91","Type":"ContainerStarted","Data":"11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39"} Sep 09 20:51:38.564891 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:38.564831 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" podStartSLOduration=2.518198488 podStartE2EDuration="8.56481438s" totalImagesPullingTime="6.046615892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:38.564792258 +0000 UTC m=+3367.189296299" watchObservedRunningTime="2026-09-09 20:51:38.56481438 +0000 UTC m=+3367.189318416" Sep 09 20:51:57.629229 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:57.629191 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerID="11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39" exitCode=0 Sep 09 20:51:57.629713 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:57.629266 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" event={"ID":"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91","Type":"ContainerDied","Data":"11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39"} Sep 09 20:51:57.629713 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:57.629594 2577 scope.go:117] "RemoveContainer" containerID="11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39" Sep 09 20:51:58.417909 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:51:58.417880 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tjbpx_must-gather-hw4jr_3d24b9db-2715-4cd4-9cb2-fbd26f68dd91/gather/0.log" Sep 09 20:52:01.919918 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:01.919886 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r5fjb_6c45f59b-2945-416b-86d6-570d211d5f28/global-pull-secret-syncer/0.log" Sep 09 20:52:02.121641 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:02.121610 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wd8dj_d82d5454-6654-4848-b291-fbb6ddc8c6ef/konnectivity-agent/0.log" Sep 09 20:52:02.215199 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:02.215120 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-99.ec2.internal_c8fd42523dcb4ea1404f0c2f6addc573/haproxy/0.log" Sep 09 20:52:03.885831 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:03.885802 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-tjbpx/must-gather-hw4jr"] Sep 09 20:52:03.886338 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:03.885975 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerName="copy" containerID="cri-o://b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd" gracePeriod=2 Sep 09 20:52:03.892545 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:03.892520 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-tjbpx/must-gather-hw4jr"] Sep 09 20:52:04.115893 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.115872 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tjbpx_must-gather-hw4jr_3d24b9db-2715-4cd4-9cb2-fbd26f68dd91/copy/0.log" Sep 09 20:52:04.117553 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.117536 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:52:04.119856 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:52:04.119830 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hw4jr\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tjbpx\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:52:04.246373 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.246347 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcl4x\" (UniqueName: \"kubernetes.io/projected/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-kube-api-access-xcl4x\") pod \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " Sep 09 20:52:04.246543 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.246438 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-must-gather-output\") pod \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\" (UID: \"3d24b9db-2715-4cd4-9cb2-fbd26f68dd91\") " Sep 09 20:52:04.247947 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.247916 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" (UID: "3d24b9db-2715-4cd4-9cb2-fbd26f68dd91"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:52:04.248744 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.248709 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-kube-api-access-xcl4x" (OuterVolumeSpecName: "kube-api-access-xcl4x") pod "3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" (UID: "3d24b9db-2715-4cd4-9cb2-fbd26f68dd91"). InnerVolumeSpecName "kube-api-access-xcl4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:52:04.347208 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.347179 2577 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-must-gather-output\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:52:04.347208 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.347203 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcl4x\" (UniqueName: \"kubernetes.io/projected/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91-kube-api-access-xcl4x\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\"" Sep 09 20:52:04.655222 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.655153 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tjbpx_must-gather-hw4jr_3d24b9db-2715-4cd4-9cb2-fbd26f68dd91/copy/0.log" Sep 09 20:52:04.655554 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.655532 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerID="b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd" exitCode=143 Sep 09 20:52:04.655635 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.655596 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:52:04.655635 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.655609 2577 scope.go:117] "RemoveContainer" containerID="b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd" Sep 09 20:52:04.658260 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:52:04.658237 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hw4jr\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tjbpx\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:52:04.663545 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.663523 2577 scope.go:117] "RemoveContainer" containerID="11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39" Sep 09 20:52:04.665879 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:52:04.665852 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hw4jr\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tjbpx\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" pod="openshift-must-gather-tjbpx/must-gather-hw4jr" Sep 09 20:52:04.676561 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.676544 2577 scope.go:117] "RemoveContainer" containerID="b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd" Sep 09 20:52:04.676860 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:52:04.676840 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd\": container with ID starting with b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd not found: ID does not exist" containerID="b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd" Sep 09 20:52:04.676909 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.676869 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd"} err="failed to get container status \"b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd\": rpc error: code = NotFound desc = could not find container \"b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd\": container with ID starting with b8a3cf5fb8f814361f642b053e02026199a567f1a6f7615592e0b65ac648b0dd not found: ID does not exist" Sep 09 20:52:04.676909 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.676885 2577 scope.go:117] "RemoveContainer" containerID="11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39" Sep 09 20:52:04.677149 ip-10-0-142-99 kubenswrapper[2577]: E0909 20:52:04.677130 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39\": container with ID starting with 11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39 not found: ID does not exist" containerID="11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39" Sep 09 20:52:04.677203 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:04.677159 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39"} err="failed to get container status \"11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39\": rpc error: code = NotFound desc = could not find container \"11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39\": container with ID starting with 11ad1a193cace303b92a321e1172d6a0d28884851264d62c442b93fc83079b39 not found: ID does not exist" Sep 09 20:52:05.879340 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:05.879306 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" path="/var/lib/kubelet/pods/3d24b9db-2715-4cd4-9cb2-fbd26f68dd91/volumes" Sep 09 20:52:06.866726 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:06.866701 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gkf5c_295d5dc3-36b0-4ed9-bed8-814a46a842c4/node-exporter/0.log" Sep 09 20:52:06.886949 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:06.886930 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gkf5c_295d5dc3-36b0-4ed9-bed8-814a46a842c4/kube-rbac-proxy/0.log" Sep 09 20:52:06.907357 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:06.907334 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gkf5c_295d5dc3-36b0-4ed9-bed8-814a46a842c4/init-textfile/0.log" Sep 09 20:52:07.121922 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.121850 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-8pk5m_a8db9e7a-2268-44a2-a2cc-2393e524524d/kube-rbac-proxy-main/0.log" Sep 09 20:52:07.146092 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.146065 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-8pk5m_a8db9e7a-2268-44a2-a2cc-2393e524524d/kube-rbac-proxy-self/0.log" Sep 09 20:52:07.170067 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.170042 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-8pk5m_a8db9e7a-2268-44a2-a2cc-2393e524524d/openshift-state-metrics/0.log" Sep 09 20:52:07.221246 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.221226 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8ae542f2-22d5-4e61-a658-a1aa6f0d9f88/prometheus/0.log" Sep 09 20:52:07.242136 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.242119 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8ae542f2-22d5-4e61-a658-a1aa6f0d9f88/config-reloader/0.log" Sep 09 20:52:07.270755 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.270738 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8ae542f2-22d5-4e61-a658-a1aa6f0d9f88/thanos-sidecar/0.log" Sep 09 20:52:07.293375 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.293349 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8ae542f2-22d5-4e61-a658-a1aa6f0d9f88/kube-rbac-proxy-web/0.log" Sep 09 20:52:07.314935 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.314916 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8ae542f2-22d5-4e61-a658-a1aa6f0d9f88/kube-rbac-proxy/0.log" Sep 09 20:52:07.339565 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.339549 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8ae542f2-22d5-4e61-a658-a1aa6f0d9f88/kube-rbac-proxy-thanos/0.log" Sep 09 20:52:07.360880 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.360865 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8ae542f2-22d5-4e61-a658-a1aa6f0d9f88/init-config-reloader/0.log" Sep 09 20:52:07.630293 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.630267 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d7b76fdd4-mfx9q_a731fbbf-ef3c-4532-8beb-f2f2adafa3bb/thanos-query/0.log" Sep 09 20:52:07.657895 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.657869 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d7b76fdd4-mfx9q_a731fbbf-ef3c-4532-8beb-f2f2adafa3bb/kube-rbac-proxy-web/0.log" Sep 09 20:52:07.682303 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.682288 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d7b76fdd4-mfx9q_a731fbbf-ef3c-4532-8beb-f2f2adafa3bb/kube-rbac-proxy/0.log" Sep 09 20:52:07.716047 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.716028 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d7b76fdd4-mfx9q_a731fbbf-ef3c-4532-8beb-f2f2adafa3bb/prom-label-proxy/0.log" Sep 09 20:52:07.743682 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.743661 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d7b76fdd4-mfx9q_a731fbbf-ef3c-4532-8beb-f2f2adafa3bb/kube-rbac-proxy-rules/0.log" Sep 09 20:52:07.774980 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:07.774952 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d7b76fdd4-mfx9q_a731fbbf-ef3c-4532-8beb-f2f2adafa3bb/kube-rbac-proxy-metrics/0.log" Sep 09 20:52:09.430683 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.430645 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/2.log" Sep 09 20:52:09.438684 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.438658 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-k8npt_6a980afb-3ae2-491e-963e-fea2f4fa2ef2/console-operator/3.log" Sep 09 20:52:09.947277 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.947233 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn"] Sep 09 20:52:09.947615 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.947598 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerName="gather" Sep 09 20:52:09.947705 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.947617 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerName="gather" Sep 09 20:52:09.947705 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.947658 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerName="copy" Sep 09 20:52:09.947705 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.947667 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerName="copy" Sep 09 20:52:09.947856 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.947752 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerName="gather" Sep 09 20:52:09.947856 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.947767 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d24b9db-2715-4cd4-9cb2-fbd26f68dd91" containerName="copy" Sep 09 20:52:09.951218 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.951199 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:09.954195 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.954172 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qzq9p\"/\"default-dockercfg-7lvqh\"" Sep 09 20:52:09.954439 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.954422 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzq9p\"/\"openshift-service-ca.crt\"" Sep 09 20:52:09.955308 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.955290 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzq9p\"/\"kube-root-ca.crt\"" Sep 09 20:52:09.964363 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:09.964324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn"] Sep 09 20:52:10.094378 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.094349 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-podres\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.094506 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.094399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6brkw\" (UniqueName: \"kubernetes.io/projected/2fd88407-8f33-4ba5-8c46-38882975ad63-kube-api-access-6brkw\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.094506 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.094474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-proc\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.094602 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.094516 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-lib-modules\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.094602 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.094542 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-sys\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195316 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6brkw\" (UniqueName: \"kubernetes.io/projected/2fd88407-8f33-4ba5-8c46-38882975ad63-kube-api-access-6brkw\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195437 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-proc\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195437 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195362 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-lib-modules\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195437 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-sys\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195437 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195411 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-proc\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195632 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-podres\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195632 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195502 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-sys\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195632 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195557 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-lib-modules\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.195632 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.195567 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2fd88407-8f33-4ba5-8c46-38882975ad63-podres\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.204098 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.204055 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6brkw\" (UniqueName: \"kubernetes.io/projected/2fd88407-8f33-4ba5-8c46-38882975ad63-kube-api-access-6brkw\") pod \"perf-node-gather-daemonset-mssjn\" (UID: \"2fd88407-8f33-4ba5-8c46-38882975ad63\") " pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.266071 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.266049 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.537218 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.537149 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn"] Sep 09 20:52:10.539974 ip-10-0-142-99 kubenswrapper[2577]: W0909 20:52:10.539947 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2fd88407_8f33_4ba5_8c46_38882975ad63.slice/crio-ea3304553fd6c764c7d6891f4dcc44c64995db892c7a152fa5712aad4ff33cac WatchSource:0}: Error finding container ea3304553fd6c764c7d6891f4dcc44c64995db892c7a152fa5712aad4ff33cac: Status 404 returned error can't find the container with id ea3304553fd6c764c7d6891f4dcc44c64995db892c7a152fa5712aad4ff33cac Sep 09 20:52:10.685078 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.685043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" event={"ID":"2fd88407-8f33-4ba5-8c46-38882975ad63","Type":"ContainerStarted","Data":"a5ef632bbf38bb0d3d33d00955964e8ad7e5fceb43b36737a15fde08a92123a5"} Sep 09 20:52:10.685192 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.685102 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" event={"ID":"2fd88407-8f33-4ba5-8c46-38882975ad63","Type":"ContainerStarted","Data":"ea3304553fd6c764c7d6891f4dcc44c64995db892c7a152fa5712aad4ff33cac"} Sep 09 20:52:10.685192 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.685125 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:10.704409 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:10.704365 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" podStartSLOduration=1.704350556 podStartE2EDuration="1.704350556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:10.703279451 +0000 UTC m=+3399.327783488" watchObservedRunningTime="2026-09-09 20:52:10.704350556 +0000 UTC m=+3399.328854630" Sep 09 20:52:11.432691 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:11.432660 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ldsrh_22ffb8c6-7119-4d56-b048-6f5111a8a5cf/dns/0.log" Sep 09 20:52:11.453085 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:11.453060 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ldsrh_22ffb8c6-7119-4d56-b048-6f5111a8a5cf/kube-rbac-proxy/0.log" Sep 09 20:52:11.576422 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:11.576369 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkj8g_c34f0e58-b333-4941-b0fb-8d0ec6c461cd/dns-node-resolver/0.log" Sep 09 20:52:12.079639 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:12.079604 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mcc67_78ec08ab-1e3b-4efc-98ed-e659284b71b6/node-ca/0.log" Sep 09 20:52:12.833019 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:12.832983 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77b88cd7dd-sz5nj_80f2786e-2c92-4c86-b816-0c0f9089b3d6/router/0.log" Sep 09 20:52:13.164760 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:13.164685 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-84qc6_0e1e15a7-89e0-4945-9a6b-1e1d97f9648a/serve-healthcheck-canary/0.log" Sep 09 20:52:13.599228 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:13.599203 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-mq4h4_bf40b6d8-3cdf-436f-b3c3-09fb46d7c383/insights-operator/0.log" Sep 09 20:52:13.601083 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:13.601056 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-mq4h4_bf40b6d8-3cdf-436f-b3c3-09fb46d7c383/insights-operator/1.log" Sep 09 20:52:13.764896 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:13.764872 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clrhp_0ba535aa-b22f-4e23-8617-971caabcb9c4/kube-rbac-proxy/0.log" Sep 09 20:52:13.783093 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:13.783074 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clrhp_0ba535aa-b22f-4e23-8617-971caabcb9c4/exporter/0.log" Sep 09 20:52:13.803411 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:13.803388 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clrhp_0ba535aa-b22f-4e23-8617-971caabcb9c4/extractor/0.log" Sep 09 20:52:16.233833 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:16.233801 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-v7h2c_c0d3cb33-b938-466c-82d8-acfbae001265/s3-init/0.log" Sep 09 20:52:16.250716 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:16.250688 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-v7h2c_c0d3cb33-b938-466c-82d8-acfbae001265/download-hf-model/0.log" Sep 09 20:52:16.699272 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:16.699237 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qzq9p/perf-node-gather-daemonset-mssjn" Sep 09 20:52:22.056211 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.056183 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g54lw_97a8e257-3a98-4fb8-b487-d2cc2a1be7bc/kube-multus-additional-cni-plugins/0.log" Sep 09 20:52:22.079068 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.079045 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g54lw_97a8e257-3a98-4fb8-b487-d2cc2a1be7bc/egress-router-binary-copy/0.log" Sep 09 20:52:22.097861 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.097836 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g54lw_97a8e257-3a98-4fb8-b487-d2cc2a1be7bc/cni-plugins/0.log" Sep 09 20:52:22.118304 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.118276 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g54lw_97a8e257-3a98-4fb8-b487-d2cc2a1be7bc/bond-cni-plugin/0.log" Sep 09 20:52:22.136114 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.136091 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g54lw_97a8e257-3a98-4fb8-b487-d2cc2a1be7bc/routeoverride-cni/0.log" Sep 09 20:52:22.157327 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.157307 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g54lw_97a8e257-3a98-4fb8-b487-d2cc2a1be7bc/whereabouts-cni-bincopy/0.log" Sep 09 20:52:22.176720 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.176695 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g54lw_97a8e257-3a98-4fb8-b487-d2cc2a1be7bc/whereabouts-cni/0.log" Sep 09 20:52:22.394303 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.394225 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-grl76_d6a74307-af6c-4b04-ab8b-ef97b1b0104c/kube-multus/0.log" Sep 09 20:52:22.443762 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.443726 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-98kdp_379c5732-e859-4f78-895c-62bd2f25915b/network-metrics-daemon/0.log" Sep 09 20:52:22.462822 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:22.462796 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-98kdp_379c5732-e859-4f78-895c-62bd2f25915b/kube-rbac-proxy/0.log" Sep 09 20:52:23.497510 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.497379 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/ovn-controller/0.log" Sep 09 20:52:23.514444 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.514423 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/ovn-acl-logging/0.log" Sep 09 20:52:23.543608 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.543579 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/kube-rbac-proxy-node/0.log" Sep 09 20:52:23.570405 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.570376 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 20:52:23.586870 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.586838 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/northd/0.log" Sep 09 20:52:23.607328 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.607310 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/nbdb/0.log" Sep 09 20:52:23.626513 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.626490 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/sbdb/0.log" Sep 09 20:52:23.671849 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:23.671827 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5rrgv_139f0b1d-e70f-458b-8a23-76e024dc6685/ovnkube-controller/0.log" Sep 09 20:52:24.908742 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:24.908713 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-wdjfl_267a61e5-1990-4b89-9026-0355c180c0ec/network-check-target-container/0.log" Sep 09 20:52:25.939938 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:25.939915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-q7w5b_ea161fd7-651e-4c61-a005-37a870b8e290/iptables-alerter/0.log" Sep 09 20:52:26.612630 ip-10-0-142-99 kubenswrapper[2577]: I0909 20:52:26.612603 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-b2f4q_3cf5b3a1-2be3-4beb-a6fd-65574667153e/tuned/0.log"