Sep 09 20:47:36.722943 ip-10-0-143-6 systemd[1]: Starting Kubernetes Kubelet... Sep 09 20:47:37.120383 ip-10-0-143-6 kubenswrapper[2538]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 20:47:37.120383 ip-10-0-143-6 kubenswrapper[2538]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 20:47:37.120383 ip-10-0-143-6 kubenswrapper[2538]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 20:47:37.120383 ip-10-0-143-6 kubenswrapper[2538]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 20:47:37.120383 ip-10-0-143-6 kubenswrapper[2538]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 20:47:37.121114 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.120441 2538 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 20:47:37.125616 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125595 2538 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:47:37.125616 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125612 2538 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:47:37.125616 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125619 2538 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:47:37.125616 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125623 2538 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125626 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125630 2538 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125633 2538 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125635 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125638 2538 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125655 2538 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125658 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125660 2538 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125663 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125666 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125670 2538 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125675 2538 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125678 2538 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125681 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125684 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125687 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125691 2538 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125694 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125696 2538 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:47:37.125801 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125699 2538 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125702 2538 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125704 2538 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125707 2538 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125712 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125715 2538 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125717 2538 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125720 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125723 2538 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125726 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125728 2538 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125731 2538 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125734 2538 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125736 2538 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125740 2538 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125743 2538 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125746 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125751 2538 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125754 2538 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125757 2538 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:47:37.126312 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125760 2538 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125762 2538 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125765 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125767 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125770 2538 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125773 2538 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125776 2538 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125778 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125781 2538 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125784 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125789 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125792 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125794 2538 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125797 2538 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125800 2538 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125803 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125806 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125808 2538 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125811 2538 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125815 2538 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:47:37.126828 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125818 2538 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125823 2538 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125826 2538 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125828 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125831 2538 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125834 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125837 2538 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125840 2538 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125843 2538 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125846 2538 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125849 2538 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125851 2538 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125854 2538 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125857 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125862 2538 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125866 2538 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125870 2538 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125873 2538 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125877 2538 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:47:37.127390 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125880 2538 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125883 2538 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125886 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125889 2538 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125892 2538 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125895 2538 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125900 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125903 2538 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125905 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125908 2538 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125911 2538 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125913 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125917 2538 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125921 2538 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125924 2538 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125928 2538 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.125931 2538 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127044 2538 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127051 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:47:37.127971 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127057 2538 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127060 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127064 2538 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127067 2538 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127070 2538 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127073 2538 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127076 2538 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127079 2538 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127082 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127084 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127090 2538 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127093 2538 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127096 2538 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127099 2538 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127102 2538 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127105 2538 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127108 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127111 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127114 2538 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:47:37.128460 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127116 2538 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127119 2538 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127122 2538 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127124 2538 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127131 2538 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127135 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127138 2538 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127141 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127143 2538 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127146 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127149 2538 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127151 2538 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127154 2538 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127157 2538 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127160 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127163 2538 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127168 2538 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127171 2538 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127173 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127176 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:47:37.128986 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127179 2538 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127181 2538 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127184 2538 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127187 2538 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127190 2538 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127192 2538 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127195 2538 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127210 2538 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127215 2538 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127219 2538 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127222 2538 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127225 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127227 2538 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127230 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127233 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127236 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127239 2538 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127242 2538 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127244 2538 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:47:37.129465 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127247 2538 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127252 2538 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127255 2538 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127258 2538 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127263 2538 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127266 2538 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127293 2538 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127296 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127419 2538 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127424 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127427 2538 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127430 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127432 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127436 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127439 2538 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127441 2538 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127444 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127447 2538 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127449 2538 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127452 2538 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:47:37.129944 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127455 2538 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127457 2538 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127460 2538 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127463 2538 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127465 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127468 2538 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127470 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127476 2538 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127479 2538 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127483 2538 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127487 2538 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127490 2538 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127493 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127496 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127499 2538 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127502 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127504 2538 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127507 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127512 2538 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:47:37.130425 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127935 2538 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127940 2538 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127944 2538 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127948 2538 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127951 2538 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127953 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127956 2538 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127958 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127961 2538 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127963 2538 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127966 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127968 2538 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127970 2538 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127973 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127976 2538 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127979 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127981 2538 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127984 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127987 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:47:37.130909 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127990 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127993 2538 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127995 2538 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.127998 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128000 2538 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128003 2538 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128006 2538 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128008 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128011 2538 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128013 2538 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128015 2538 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128018 2538 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128021 2538 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128024 2538 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128026 2538 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128028 2538 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128031 2538 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128033 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128036 2538 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128039 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:47:37.131376 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128041 2538 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128044 2538 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128046 2538 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128048 2538 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128051 2538 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128053 2538 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128056 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128059 2538 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128061 2538 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128064 2538 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128068 2538 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128071 2538 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128073 2538 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128075 2538 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128078 2538 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128080 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128083 2538 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128085 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128088 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128090 2538 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:47:37.131902 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128092 2538 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128095 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128098 2538 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128102 2538 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128105 2538 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128108 2538 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128111 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128113 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128117 2538 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128119 2538 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128122 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128124 2538 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128127 2538 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128130 2538 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128133 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128136 2538 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128139 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128142 2538 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128144 2538 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:47:37.132396 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128147 2538 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128149 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128151 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128155 2538 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128158 2538 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128160 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128163 2538 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128166 2538 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128168 2538 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128170 2538 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128173 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128175 2538 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128178 2538 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128180 2538 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128183 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128185 2538 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128187 2538 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128190 2538 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128192 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128195 2538 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:47:37.132891 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128197 2538 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128272 2538 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128304 2538 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128310 2538 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128314 2538 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128318 2538 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128321 2538 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128325 2538 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128329 2538 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128332 2538 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128334 2538 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128338 2538 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128341 2538 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128344 2538 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128346 2538 flags.go:64] FLAG: --cgroup-root="" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128350 2538 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128353 2538 flags.go:64] FLAG: --client-ca-file="" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128357 2538 flags.go:64] FLAG: --cloud-provider="external" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128359 2538 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128363 2538 flags.go:64] FLAG: --cluster-domain="" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128366 2538 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128369 2538 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128371 2538 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128374 2538 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 20:47:37.133409 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128378 2538 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128381 2538 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128383 2538 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128386 2538 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128389 2538 flags.go:64] FLAG: --contention-profiling="false" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128393 2538 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128395 2538 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128398 2538 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128401 2538 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128409 2538 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128412 2538 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128415 2538 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128418 2538 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128420 2538 flags.go:64] FLAG: --enable-server="true" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128423 2538 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128427 2538 flags.go:64] FLAG: --event-burst="100" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128434 2538 flags.go:64] FLAG: --event-qps="50" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128437 2538 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128439 2538 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128442 2538 flags.go:64] FLAG: --eviction-hard="" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128446 2538 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128448 2538 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128451 2538 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128455 2538 flags.go:64] FLAG: --eviction-soft="" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128458 2538 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 20:47:37.134001 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128461 2538 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128464 2538 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128467 2538 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128470 2538 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128473 2538 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128476 2538 flags.go:64] FLAG: --feature-gates="" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128479 2538 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128482 2538 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128485 2538 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128488 2538 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128491 2538 flags.go:64] FLAG: --healthz-port="10248" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128493 2538 flags.go:64] FLAG: --help="false" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128497 2538 flags.go:64] FLAG: --hostname-override="ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128500 2538 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128503 2538 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128506 2538 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128509 2538 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128512 2538 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128515 2538 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128518 2538 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128520 2538 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128523 2538 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128526 2538 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128529 2538 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 20:47:37.134658 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128532 2538 flags.go:64] FLAG: --kube-reserved="" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128534 2538 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128537 2538 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128540 2538 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128542 2538 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128545 2538 flags.go:64] FLAG: --lock-file="" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128549 2538 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128552 2538 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128555 2538 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128560 2538 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128563 2538 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128566 2538 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128568 2538 flags.go:64] FLAG: --logging-format="text" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128571 2538 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128574 2538 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128577 2538 flags.go:64] FLAG: --manifest-url="" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128579 2538 flags.go:64] FLAG: --manifest-url-header="" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128583 2538 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128586 2538 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128590 2538 flags.go:64] FLAG: --max-pods="110" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128593 2538 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128596 2538 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128599 2538 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128602 2538 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128604 2538 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 20:47:37.135245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128607 2538 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128611 2538 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128617 2538 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128620 2538 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128623 2538 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128625 2538 flags.go:64] FLAG: --pod-cidr="" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128628 2538 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128633 2538 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128636 2538 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128654 2538 flags.go:64] FLAG: --pods-per-core="0" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128657 2538 flags.go:64] FLAG: --port="10250" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128660 2538 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128663 2538 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-017bcc59ea1db86f5" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128668 2538 flags.go:64] FLAG: --qos-reserved="" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128671 2538 flags.go:64] FLAG: --read-only-port="10255" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128674 2538 flags.go:64] FLAG: --register-node="true" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128676 2538 flags.go:64] FLAG: --register-with-taints="" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128680 2538 flags.go:64] FLAG: --registry-burst="10" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128684 2538 flags.go:64] FLAG: --registry-qps="5" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128686 2538 flags.go:64] FLAG: --reserved-cpus="" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128689 2538 flags.go:64] FLAG: --reserved-memory="" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128692 2538 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128696 2538 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128699 2538 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128701 2538 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 20:47:37.135886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128704 2538 flags.go:64] FLAG: --runonce="false" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128707 2538 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128710 2538 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128714 2538 flags.go:64] FLAG: --seccomp-default="false" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128717 2538 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128719 2538 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128722 2538 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128725 2538 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128728 2538 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128730 2538 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128733 2538 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128736 2538 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128739 2538 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128741 2538 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128744 2538 flags.go:64] FLAG: --system-cgroups="" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128747 2538 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128752 2538 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128755 2538 flags.go:64] FLAG: --tls-cert-file="" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128757 2538 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128761 2538 flags.go:64] FLAG: --tls-min-version="" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128765 2538 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128767 2538 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128770 2538 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128773 2538 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128776 2538 flags.go:64] FLAG: --v="2" Sep 09 20:47:37.136488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128779 2538 flags.go:64] FLAG: --version="false" Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128784 2538 flags.go:64] FLAG: --vmodule="" Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128787 2538 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.128790 2538 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128902 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128906 2538 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128908 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128911 2538 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128913 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128916 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128920 2538 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128922 2538 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128925 2538 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128927 2538 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128930 2538 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128933 2538 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128935 2538 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128937 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128940 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128942 2538 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128945 2538 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:47:37.137114 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128948 2538 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128950 2538 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128952 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128955 2538 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128957 2538 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128960 2538 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128964 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128966 2538 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128968 2538 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128971 2538 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128973 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128976 2538 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128978 2538 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128981 2538 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128983 2538 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128986 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128989 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128991 2538 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128994 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.128997 2538 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:47:37.137620 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129000 2538 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129004 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129006 2538 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129008 2538 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129011 2538 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129013 2538 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129016 2538 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129020 2538 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129023 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129026 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129029 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129032 2538 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129035 2538 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129038 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129041 2538 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129044 2538 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129047 2538 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129050 2538 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129054 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:47:37.138163 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129056 2538 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129059 2538 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129062 2538 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129065 2538 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129067 2538 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129070 2538 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129072 2538 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129075 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129077 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129080 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129082 2538 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129085 2538 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129087 2538 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129089 2538 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129094 2538 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129096 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129099 2538 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129102 2538 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129104 2538 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:47:37.138619 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129107 2538 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129111 2538 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129113 2538 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129116 2538 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129118 2538 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129120 2538 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129123 2538 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129125 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129128 2538 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129130 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129132 2538 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129135 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129138 2538 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129140 2538 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129143 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129145 2538 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129148 2538 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129150 2538 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129153 2538 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129156 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:47:37.139098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129158 2538 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129162 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129164 2538 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.129167 2538 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.129859 2538 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.136796 2538 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.136809 2538 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.138533 2538 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.138540 2538 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138592 2538 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138597 2538 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138601 2538 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138604 2538 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138607 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138610 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:47:37.139605 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138613 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138615 2538 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138618 2538 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138621 2538 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138624 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138627 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138630 2538 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138633 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138637 2538 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138652 2538 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138655 2538 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138658 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138661 2538 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138663 2538 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138666 2538 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138669 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138671 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138674 2538 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138678 2538 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:47:37.140089 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138680 2538 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138683 2538 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138685 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138688 2538 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138691 2538 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138694 2538 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138696 2538 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138699 2538 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138702 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138704 2538 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138707 2538 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138710 2538 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138712 2538 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138714 2538 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138717 2538 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138719 2538 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138722 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138724 2538 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138727 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138729 2538 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:47:37.140558 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138732 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138735 2538 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138737 2538 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138740 2538 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138742 2538 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138744 2538 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138747 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138749 2538 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138752 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138755 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138757 2538 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138760 2538 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138763 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138766 2538 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138770 2538 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138774 2538 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138779 2538 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138782 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138785 2538 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:47:37.141050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138788 2538 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138791 2538 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138793 2538 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138796 2538 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138798 2538 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138800 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138803 2538 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138805 2538 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138808 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138810 2538 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138813 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138815 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138818 2538 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138820 2538 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138823 2538 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138826 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138828 2538 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138831 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138833 2538 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138836 2538 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:47:37.141540 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138838 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138841 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138843 2538 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138845 2538 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138848 2538 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138850 2538 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138853 2538 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138856 2538 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138858 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138861 2538 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138864 2538 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138866 2538 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138869 2538 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138871 2538 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138874 2538 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:47:37.142071 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.138879 2538 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138990 2538 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138995 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.138998 2538 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139000 2538 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139003 2538 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139006 2538 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139008 2538 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139010 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139013 2538 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139015 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139018 2538 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139020 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139022 2538 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139025 2538 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139027 2538 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139030 2538 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139032 2538 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139035 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139038 2538 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:47:37.142516 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139041 2538 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139043 2538 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139045 2538 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139048 2538 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139051 2538 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139053 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139056 2538 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139058 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139061 2538 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139063 2538 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139066 2538 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139068 2538 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139071 2538 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139074 2538 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139078 2538 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139080 2538 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139084 2538 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139086 2538 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139089 2538 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139092 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:47:37.143112 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139094 2538 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139097 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139100 2538 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139103 2538 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139105 2538 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139108 2538 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139111 2538 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139114 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139116 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139119 2538 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139122 2538 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139124 2538 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139127 2538 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139129 2538 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139132 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139135 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139137 2538 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139141 2538 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139144 2538 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139146 2538 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:47:37.143602 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139149 2538 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139152 2538 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139154 2538 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139157 2538 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139159 2538 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139161 2538 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139164 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139167 2538 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139170 2538 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139173 2538 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139176 2538 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139179 2538 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139181 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139184 2538 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139187 2538 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139190 2538 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139192 2538 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139195 2538 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139204 2538 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:47:37.144098 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139207 2538 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139210 2538 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139212 2538 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139214 2538 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139217 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139220 2538 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139222 2538 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139225 2538 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139227 2538 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139230 2538 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139232 2538 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139235 2538 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139238 2538 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139241 2538 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139243 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139245 2538 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139248 2538 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139250 2538 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139253 2538 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139255 2538 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:47:37.144562 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:37.139258 2538 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:47:37.145052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.139262 2538 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 20:47:37.145052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.139361 2538 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 20:47:37.145052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.141791 2538 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 20:47:37.145052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.142788 2538 server.go:1019] "Starting client certificate rotation" Sep 09 20:47:37.145052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.142892 2538 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 20:47:37.145052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.144076 2538 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 20:47:37.145230 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.145176 2538 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 20:47:37.148091 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.148074 2538 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 20:47:37.157445 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.157429 2538 log.go:25] "Validated CRI v1 runtime API" Sep 09 20:47:37.165077 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.165061 2538 log.go:25] "Validated CRI v1 image API" Sep 09 20:47:37.166041 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.166024 2538 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 20:47:37.169723 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.169697 2538 fs.go:135] Filesystem UUIDs: map[18bcc9fc-97c9-407d-a3ac-88e51fb3282d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cd63e12a-db55-4032-a9d8-6033206bc705:/dev/nvme0n1p3] Sep 09 20:47:37.169723 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.169722 2538 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 20:47:37.176482 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.176366 2538 manager.go:216] Machine: {Timestamp:2026-09-09 20:47:37.173661231 +0000 UTC m=+0.338901828 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098276 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24a4cf57a83c60027f9080982c2226 SystemUUID:ec24a4cf-57a8-3c60-027f-9080982c2226 BootID:632bf5d1-195e-4f5c-8cf6-2ae0a4567398 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8e:45:87:5d:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8e:45:87:5d:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:72:ec:1a:08:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 20:47:37.176482 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.176474 2538 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 20:47:37.176603 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.176551 2538 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 20:47:37.177977 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.177962 2538 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 20:47:37.179449 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.179424 2538 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 20:47:37.179573 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.179451 2538 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 20:47:37.180121 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.180109 2538 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 20:47:37.180121 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.180122 2538 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 20:47:37.180201 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.180129 2538 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 20:47:37.180851 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.180841 2538 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 20:47:37.182010 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.182001 2538 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 20:47:37.185801 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.185791 2538 state_mem.go:36] "Initialized new in-memory state store" Sep 09 20:47:37.185907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.185898 2538 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 20:47:37.185946 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.185928 2538 kubelet.go:486] "Attempting to sync node with API server" Sep 09 20:47:37.185946 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.185938 2538 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 20:47:37.186000 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.185953 2538 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 20:47:37.186000 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.185962 2538 kubelet.go:398] "Adding apiserver pod source" Sep 09 20:47:37.186000 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.185970 2538 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 20:47:37.186357 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.186344 2538 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 20:47:37.186396 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.186373 2538 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 20:47:37.188781 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.188764 2538 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 20:47:37.191177 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.191158 2538 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 20:47:37.191273 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.191195 2538 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 20:47:37.192054 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192041 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 20:47:37.192100 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192061 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 20:47:37.192100 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192068 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 20:47:37.192100 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192073 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 20:47:37.192100 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192082 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 20:47:37.192100 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192087 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 20:47:37.192100 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192100 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 20:47:37.192276 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192105 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 20:47:37.192276 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192111 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 20:47:37.192276 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192117 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 20:47:37.192276 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192130 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 20:47:37.192276 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192135 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 20:47:37.192847 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192828 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 20:47:37.192847 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.192837 2538 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 20:47:37.196278 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.196264 2538 server.go:1268] "Started kubelet" Sep 09 20:47:37.196402 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.196358 2538 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 20:47:37.196459 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.196423 2538 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 20:47:37.196659 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.196491 2538 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 20:47:37.197081 ip-10-0-143-6 systemd[1]: Started Kubernetes Kubelet. Sep 09 20:47:37.197608 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.197589 2538 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 20:47:37.197810 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.197790 2538 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 20:47:37.197952 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.197918 2538 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 20:47:37.197952 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.197930 2538 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 20:47:37.202870 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.202845 2538 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 20:47:37.203403 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.203387 2538 server.go:310] "Adding debug handlers to kubelet server" Sep 09 20:47:37.205707 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.205687 2538 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 20:47:37.205707 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.205698 2538 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 20:47:37.205843 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.205830 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.206377 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.206320 2538 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 20:47:37.206377 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.206333 2538 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 20:47:37.206377 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.206349 2538 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 20:47:37.206544 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.205166 2538 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-6.ec2.internal.18d3c21173062a9e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-6.ec2.internal,UID:ip-10-0-143-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-6.ec2.internal,},FirstTimestamp:2026-09-09 20:47:37.196235422 +0000 UTC m=+0.361476023,LastTimestamp:2026-09-09 20:47:37.196235422 +0000 UTC m=+0.361476023,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-6.ec2.internal,}" Sep 09 20:47:37.206544 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.206419 2538 reconstruct.go:97] "Volume reconstruction finished" Sep 09 20:47:37.206544 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.206430 2538 reconciler.go:29] "Reconciler: start to sync state" Sep 09 20:47:37.207415 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.207396 2538 factory.go:55] Registering systemd factory Sep 09 20:47:37.207493 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.207419 2538 factory.go:223] Registration of the systemd container factory successfully Sep 09 20:47:37.207670 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.207635 2538 factory.go:153] Registering CRI-O factory Sep 09 20:47:37.207744 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.207673 2538 factory.go:223] Registration of the crio container factory successfully Sep 09 20:47:37.207744 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.207727 2538 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 20:47:37.207837 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.207745 2538 factory.go:103] Registering Raw factory Sep 09 20:47:37.207837 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.207760 2538 manager.go:1196] Started watching for new ooms in manager Sep 09 20:47:37.208176 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.208160 2538 manager.go:349] Starting recovery of all containers Sep 09 20:47:37.218692 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.218664 2538 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 20:47:37.218692 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.218671 2538 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 20:47:37.219333 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.219318 2538 manager.go:354] Recovery completed Sep 09 20:47:37.227401 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.227380 2538 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vt8tj" Sep 09 20:47:37.230186 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.230174 2538 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:47:37.232110 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.232096 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:47:37.232158 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.232124 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:47:37.232158 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.232137 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:47:37.232255 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.232215 2538 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vt8tj" Sep 09 20:47:37.233154 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.233140 2538 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 20:47:37.233154 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.233150 2538 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:47:37.233332 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.233152 2538 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 20:47:37.233332 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.233232 2538 state_mem.go:36] "Initialized new in-memory state store" Sep 09 20:47:37.233826 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.233809 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:47:37.233887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.233833 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:47:37.233887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.233843 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:47:37.234282 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.234270 2538 policy_none.go:49] "None policy: Start" Sep 09 20:47:37.234313 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.234287 2538 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 20:47:37.234313 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.234298 2538 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 20:47:37.235309 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.235298 2538 policy_none.go:47] "Start" Sep 09 20:47:37.276263 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.276246 2538 manager.go:337] "Starting Device Plugin manager" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.276284 2538 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.276297 2538 server.go:87] "Starting device plugin registration server" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.276596 2538 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.276607 2538 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.276740 2538 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.276815 2538 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.276824 2538 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.277252 2538 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 20:47:37.288882 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.277305 2538 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.316045 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.316015 2538 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 20:47:37.317431 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.317411 2538 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 20:47:37.317505 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.317458 2538 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 20:47:37.317505 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.317478 2538 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 20:47:37.317569 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.317522 2538 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 20:47:37.320137 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.320117 2538 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:47:37.377210 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.377102 2538 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:47:37.378146 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.378129 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:47:37.378209 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.378160 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:47:37.378209 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.378171 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:47:37.378209 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.378195 2538 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.390186 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.390169 2538 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.390266 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.390193 2538 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-6.ec2.internal\": node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.413187 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.413171 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.417675 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.417638 2538 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"] Sep 09 20:47:37.417728 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.417719 2538 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:47:37.419444 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.419430 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:47:37.419489 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.419462 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:47:37.419489 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.419472 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:47:37.420723 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.420711 2538 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:47:37.420860 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.420845 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.420893 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.420879 2538 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:47:37.421511 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.421494 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:47:37.421582 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.421525 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:47:37.421582 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.421539 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:47:37.421582 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.421494 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:47:37.421741 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.421598 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:47:37.421741 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.421611 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:47:37.422856 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.422841 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.422910 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.422869 2538 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:47:37.423578 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.423559 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:47:37.423677 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.423584 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:47:37.423677 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.423595 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:47:37.446253 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.446234 2538 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-6.ec2.internal\" not found" node="ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.450667 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.450637 2538 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-6.ec2.internal\" not found" node="ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.508795 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.508774 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80bed961383d1f2e370365643dbe1e93-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"80bed961383d1f2e370365643dbe1e93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.508865 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.508798 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d66a9a646305fb874ccf05b3f9514ed5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"d66a9a646305fb874ccf05b3f9514ed5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.508865 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.508815 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d66a9a646305fb874ccf05b3f9514ed5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"d66a9a646305fb874ccf05b3f9514ed5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.513899 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.513886 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.609912 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.609888 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80bed961383d1f2e370365643dbe1e93-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"80bed961383d1f2e370365643dbe1e93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.610006 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.609917 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d66a9a646305fb874ccf05b3f9514ed5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"d66a9a646305fb874ccf05b3f9514ed5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.610006 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.609935 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d66a9a646305fb874ccf05b3f9514ed5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"d66a9a646305fb874ccf05b3f9514ed5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.610006 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.610001 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d66a9a646305fb874ccf05b3f9514ed5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"d66a9a646305fb874ccf05b3f9514ed5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.610125 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.610003 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80bed961383d1f2e370365643dbe1e93-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"80bed961383d1f2e370365643dbe1e93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.610125 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.609999 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d66a9a646305fb874ccf05b3f9514ed5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"d66a9a646305fb874ccf05b3f9514ed5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.613957 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.613941 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.714720 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.714658 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.748840 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.748819 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.753635 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:37.753623 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Sep 09 20:47:37.815298 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.815278 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:37.915762 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:37.915740 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.016244 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.016220 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.116797 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.116769 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.142927 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.142908 2538 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 20:47:38.143301 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.143026 2538 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 20:47:38.217696 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.217672 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.234432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.234396 2538 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:42:38 +0000 UTC" deadline="2028-04-04 03:09:31.166511595 +0000 UTC" Sep 09 20:47:38.234432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.234424 2538 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13734h21m52.932090883s" Sep 09 20:47:38.272827 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:38.272768 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66a9a646305fb874ccf05b3f9514ed5.slice/crio-3ee69c46d90c0ebe40cf857018d7bddf065146e1715dd2c567e5cb7e34121000 WatchSource:0}: Error finding container 3ee69c46d90c0ebe40cf857018d7bddf065146e1715dd2c567e5cb7e34121000: Status 404 returned error can't find the container with id 3ee69c46d90c0ebe40cf857018d7bddf065146e1715dd2c567e5cb7e34121000 Sep 09 20:47:38.273289 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:38.273268 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80bed961383d1f2e370365643dbe1e93.slice/crio-63c351427b6c418370e69d9bcbd6abc679a46f069c678bd43e6fabde096d815a WatchSource:0}: Error finding container 63c351427b6c418370e69d9bcbd6abc679a46f069c678bd43e6fabde096d815a: Status 404 returned error can't find the container with id 63c351427b6c418370e69d9bcbd6abc679a46f069c678bd43e6fabde096d815a Sep 09 20:47:38.277319 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.277303 2538 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 20:47:38.318617 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.318591 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.320555 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.320504 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"d66a9a646305fb874ccf05b3f9514ed5","Type":"ContainerStarted","Data":"3ee69c46d90c0ebe40cf857018d7bddf065146e1715dd2c567e5cb7e34121000"} Sep 09 20:47:38.321566 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.321542 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" event={"ID":"80bed961383d1f2e370365643dbe1e93","Type":"ContainerStarted","Data":"63c351427b6c418370e69d9bcbd6abc679a46f069c678bd43e6fabde096d815a"} Sep 09 20:47:38.363042 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.363022 2538 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:47:38.419253 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.419231 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.500783 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:38.500756 2538 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:47:38.520254 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.520218 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.620759 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.620683 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:38.721505 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:38.721471 2538 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Sep 09 20:47:40.339575 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.339413 2538 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:47:40.352888 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.352863 2538 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Sep 09 20:47:40.365277 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.365251 2538 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 20:47:40.366441 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.366419 2538 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Sep 09 20:47:40.375517 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.375494 2538 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 20:47:40.717137 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.717105 2538 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:47:40.733612 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.733589 2538 apiserver.go:52] "Watching apiserver" Sep 09 20:47:40.742586 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.742564 2538 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 20:47:40.744458 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.744432 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-wjgh2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal","openshift-network-diagnostics/network-check-target-5kl9j","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf","openshift-cluster-node-tuning-operator/tuned-9m89j","openshift-image-registry/node-ca-vgpth","openshift-multus/multus-additional-cni-plugins-h64f5","openshift-multus/multus-rnlpl","openshift-multus/network-metrics-daemon-84ccl","openshift-network-operator/iptables-alerter-mchtr","openshift-ovn-kubernetes/ovnkube-node-t98tr","kube-system/global-pull-secret-syncer-fd8jq","kube-system/konnectivity-agent-dcdj7","kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"] Sep 09 20:47:40.746008 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.745988 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.747158 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.747133 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.748153 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.748138 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:40.748269 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.748191 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:40.749025 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.749008 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vb7gd\"" Sep 09 20:47:40.749104 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.749007 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:47:40.749104 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.749051 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 20:47:40.749104 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.749065 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 20:47:40.749756 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.749580 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.750908 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.750884 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:40.752394 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.752293 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.753655 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.753587 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 20:47:40.753655 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.753592 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 20:47:40.753857 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.753842 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 20:47:40.754015 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.753999 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b65vl\"" Sep 09 20:47:40.754149 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.754137 2538 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 20:47:40.754216 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.754174 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 20:47:40.754295 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.754261 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xw5fb\"" Sep 09 20:47:40.754355 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.754343 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 20:47:40.754528 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.754515 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-glpps\"" Sep 09 20:47:40.754616 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.754583 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 20:47:40.754673 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.754634 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 20:47:40.755410 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.755386 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:40.755509 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.755458 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:40.756108 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.756084 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 20:47:40.756199 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.756160 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 20:47:40.756667 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.756650 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.758163 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.758124 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.758524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.758505 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 20:47:40.758610 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.758512 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rsct6\"" Sep 09 20:47:40.760915 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.759950 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.761214 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.761193 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 20:47:40.761542 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.761509 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.761626 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.761574 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:40.762825 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.762803 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.762927 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.762876 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 20:47:40.764269 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.764248 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 20:47:40.764573 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.764553 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wvgpm\"" Sep 09 20:47:40.764714 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.764699 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 20:47:40.764786 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.764770 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 20:47:40.764837 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.764789 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 20:47:40.765545 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.764975 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 20:47:40.765545 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765156 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 20:47:40.765545 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765298 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 20:47:40.765545 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765329 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ghr7z\"" Sep 09 20:47:40.765545 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765334 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:47:40.765545 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765342 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 20:47:40.765545 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765395 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765935 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0933970-91fd-4048-b522-eaba643670bb-host\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765974 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-socket-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.765946 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nwg47\"" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766034 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-registration-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766148 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766186 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766261 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-sys-fs\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766293 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62a757be-03e0-47f8-9cd4-efa063d271a9-host-slash\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766346 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llzd7\" (UniqueName: \"kubernetes.io/projected/62a757be-03e0-47f8-9cd4-efa063d271a9-kube-api-access-llzd7\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.766398 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766370 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/721b84f2-9419-4d85-bc3f-d3ed7f94d9aa-agent-certs\") pod \"konnectivity-agent-dcdj7\" (UID: \"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa\") " pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766425 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766463 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-etc-selinux\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766490 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/62a757be-03e0-47f8-9cd4-efa063d271a9-iptables-alerter-script\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766520 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03b239fd-3642-4f5d-9380-8c7823fd82ed-hosts-file\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766542 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766570 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-device-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766604 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766614 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-54c4s\"" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766627 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03b239fd-3642-4f5d-9380-8c7823fd82ed-tmp-dir\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766666 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0933970-91fd-4048-b522-eaba643670bb-serviceca\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766688 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xp6c\" (UniqueName: \"kubernetes.io/projected/f0933970-91fd-4048-b522-eaba643670bb-kube-api-access-6xp6c\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766738 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5nl4\" (UniqueName: \"kubernetes.io/projected/7cc5b2fa-35fe-47b7-836b-33e555c99943-kube-api-access-v5nl4\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766763 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79s2p\" (UniqueName: \"kubernetes.io/projected/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-kube-api-access-79s2p\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766804 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c82d6\" (UniqueName: \"kubernetes.io/projected/03b239fd-3642-4f5d-9380-8c7823fd82ed-kube-api-access-c82d6\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.767067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.766826 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/721b84f2-9419-4d85-bc3f-d3ed7f94d9aa-konnectivity-ca\") pod \"konnectivity-agent-dcdj7\" (UID: \"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa\") " pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:40.783783 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.783757 2538 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 20:47:40.807488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.807467 2538 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d25ch" Sep 09 20:47:40.816275 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.816256 2538 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d25ch" Sep 09 20:47:40.853737 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.853706 2538 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 20:47:40.866988 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.866963 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-netns\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.867095 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.866999 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5nl4\" (UniqueName: \"kubernetes.io/projected/7cc5b2fa-35fe-47b7-836b-33e555c99943-kube-api-access-v5nl4\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.867095 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867020 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-run-netns\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867095 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867042 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-multus-certs\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.867095 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867064 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-etc-kubernetes\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.867335 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867107 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzd7\" (UniqueName: \"kubernetes.io/projected/62a757be-03e0-47f8-9cd4-efa063d271a9-kube-api-access-llzd7\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.867335 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867151 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79d5j\" (UniqueName: \"kubernetes.io/projected/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-kube-api-access-79d5j\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.867335 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867170 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-slash\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867344 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-run-ovn-kubernetes\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867379 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-cni-netd\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867414 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb4nz\" (UniqueName: \"kubernetes.io/projected/f41571ca-a223-43b9-bed4-f4975da190db-kube-api-access-wb4nz\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867439 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76314433-6fa3-4bbe-835a-075283005f46-cni-binary-copy\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.867621 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867471 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-etc-selinux\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.867621 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867497 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.867621 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867522 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-kubernetes\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.867621 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867544 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/291d4321-4e22-4921-a84c-707139c9b66a-kubelet-config\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.867621 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867565 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-node-log\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867621 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867601 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-etc-selinux\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.867846 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867638 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-cni-multus\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.867846 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867668 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0933970-91fd-4048-b522-eaba643670bb-serviceca\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.867846 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867684 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-ovnkube-config\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867846 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867730 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79s2p\" (UniqueName: \"kubernetes.io/projected/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-kube-api-access-79s2p\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:40.867846 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867759 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c82d6\" (UniqueName: \"kubernetes.io/projected/03b239fd-3642-4f5d-9380-8c7823fd82ed-kube-api-access-c82d6\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.867846 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867782 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-ovn\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.867846 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867817 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867868 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-sys-fs\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867897 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/721b84f2-9419-4d85-bc3f-d3ed7f94d9aa-agent-certs\") pod \"konnectivity-agent-dcdj7\" (UID: \"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa\") " pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867922 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867949 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysctl-conf\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867958 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-sys-fs\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867971 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.867995 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-env-overrides\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868023 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-cni-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868047 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cnibin\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868091 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-system-cni-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868124 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0933970-91fd-4048-b522-eaba643670bb-serviceca\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868137 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-cnibin\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868154 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-cni-bin\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868173 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-modprobe-d\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.868196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868194 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-var-lib-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868217 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868265 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/721b84f2-9419-4d85-bc3f-d3ed7f94d9aa-konnectivity-ca\") pod \"konnectivity-agent-dcdj7\" (UID: \"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa\") " pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868374 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0933970-91fd-4048-b522-eaba643670bb-host\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868405 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-kubelet\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868431 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-systemd-units\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868436 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0933970-91fd-4048-b522-eaba643670bb-host\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868455 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-cni-bin\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868483 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gvbk\" (UniqueName: \"kubernetes.io/projected/76314433-6fa3-4bbe-835a-075283005f46-kube-api-access-9gvbk\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868507 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-systemd\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868529 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-socket-dir-parent\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868559 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-hostroot\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868585 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868609 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03b239fd-3642-4f5d-9380-8c7823fd82ed-hosts-file\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868632 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ptvl\" (UniqueName: \"kubernetes.io/projected/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-kube-api-access-9ptvl\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868690 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03b239fd-3642-4f5d-9380-8c7823fd82ed-hosts-file\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868696 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysconfig\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.868903 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868746 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-run\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868756 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/721b84f2-9419-4d85-bc3f-d3ed7f94d9aa-konnectivity-ca\") pod \"konnectivity-agent-dcdj7\" (UID: \"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa\") " pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868774 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868772 2538 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868803 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-device-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868809 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868829 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03b239fd-3642-4f5d-9380-8c7823fd82ed-tmp-dir\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868849 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-device-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868852 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xp6c\" (UniqueName: \"kubernetes.io/projected/f0933970-91fd-4048-b522-eaba643670bb-kube-api-access-6xp6c\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868884 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-etc-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868905 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-conf-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.868931 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cni-binary-copy\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869117 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysctl-d\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869125 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03b239fd-3642-4f5d-9380-8c7823fd82ed-tmp-dir\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869140 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-sys\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869183 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-lib-modules\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869205 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-var-lib-kubelet\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.869662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869247 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-socket-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869277 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-registration-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869343 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-registration-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869360 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62a757be-03e0-47f8-9cd4-efa063d271a9-host-slash\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869393 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-system-cni-dir\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869415 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7cc5b2fa-35fe-47b7-836b-33e555c99943-socket-dir\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869417 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869447 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62a757be-03e0-47f8-9cd4-efa063d271a9-host-slash\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869452 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-host\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869497 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-log-socket\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869520 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f41571ca-a223-43b9-bed4-f4975da190db-ovn-node-metrics-cert\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869544 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/62a757be-03e0-47f8-9cd4-efa063d271a9-iptables-alerter-script\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869563 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-systemd\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869583 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-tuned\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869614 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/291d4321-4e22-4921-a84c-707139c9b66a-dbus\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869634 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-ovnkube-script-lib\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.870343 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869654 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-k8s-cni-cncf-io\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.870880 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869673 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-kubelet\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.870880 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869691 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76314433-6fa3-4bbe-835a-075283005f46-multus-daemon-config\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.870880 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869714 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:40.870880 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.869731 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-tmp\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.870880 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.869840 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:40.870880 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.869932 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:41.369896931 +0000 UTC m=+2.988557141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:40.870880 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.870086 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/62a757be-03e0-47f8-9cd4-efa063d271a9-iptables-alerter-script\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.873290 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.873273 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/721b84f2-9419-4d85-bc3f-d3ed7f94d9aa-agent-certs\") pod \"konnectivity-agent-dcdj7\" (UID: \"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa\") " pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:40.880646 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.880613 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:47:40.880646 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.880638 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:47:40.880849 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.880652 2538 projected.go:196] Error preparing data for projected volume kube-api-access-fbwmm for pod openshift-network-diagnostics/network-check-target-5kl9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:40.880849 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.880707 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm podName:c6949849-ef14-4726-9cbf-6c5fce30374e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:41.380691221 +0000 UTC m=+2.999351454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwmm" (UniqueName: "kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm") pod "network-check-target-5kl9j" (UID: "c6949849-ef14-4726-9cbf-6c5fce30374e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:40.884388 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.884367 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xp6c\" (UniqueName: \"kubernetes.io/projected/f0933970-91fd-4048-b522-eaba643670bb-kube-api-access-6xp6c\") pod \"node-ca-vgpth\" (UID: \"f0933970-91fd-4048-b522-eaba643670bb\") " pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:40.884649 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.884623 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c82d6\" (UniqueName: \"kubernetes.io/projected/03b239fd-3642-4f5d-9380-8c7823fd82ed-kube-api-access-c82d6\") pod \"node-resolver-wjgh2\" (UID: \"03b239fd-3642-4f5d-9380-8c7823fd82ed\") " pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:40.884937 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.884909 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llzd7\" (UniqueName: \"kubernetes.io/projected/62a757be-03e0-47f8-9cd4-efa063d271a9-kube-api-access-llzd7\") pod \"iptables-alerter-mchtr\" (UID: \"62a757be-03e0-47f8-9cd4-efa063d271a9\") " pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:40.885671 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.885632 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5nl4\" (UniqueName: \"kubernetes.io/projected/7cc5b2fa-35fe-47b7-836b-33e555c99943-kube-api-access-v5nl4\") pod \"aws-ebs-csi-driver-node-gw4qf\" (UID: \"7cc5b2fa-35fe-47b7-836b-33e555c99943\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:40.887285 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.887265 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79s2p\" (UniqueName: \"kubernetes.io/projected/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-kube-api-access-79s2p\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:40.970721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970647 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ptvl\" (UniqueName: \"kubernetes.io/projected/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-kube-api-access-9ptvl\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.970721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970685 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysconfig\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.970721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970711 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-run\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.970969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970735 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-etc-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.970969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970783 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-etc-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.970969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970837 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysconfig\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.970969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970910 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-conf-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.970969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970947 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cni-binary-copy\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.970969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970962 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-run\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.970974 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysctl-d\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971001 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-sys\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971002 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-conf-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971019 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-lib-modules\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971034 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-var-lib-kubelet\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971050 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-system-cni-dir\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971065 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971084 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-host\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971101 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-log-socket\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971105 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysctl-d\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971125 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f41571ca-a223-43b9-bed4-f4975da190db-ovn-node-metrics-cert\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971150 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-systemd\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971146 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-system-cni-dir\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971174 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-tuned\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971193 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/291d4321-4e22-4921-a84c-707139c9b66a-dbus\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971212 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-sys\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.971329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971215 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-ovnkube-script-lib\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971270 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-k8s-cni-cncf-io\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971294 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-kubelet\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971317 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76314433-6fa3-4bbe-835a-075283005f46-multus-daemon-config\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971363 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-tmp\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971387 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-netns\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971412 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-run-netns\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971436 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-multus-certs\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971469 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-etc-kubernetes\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971493 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79d5j\" (UniqueName: \"kubernetes.io/projected/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-kube-api-access-79d5j\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971517 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-slash\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971541 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-run-ovn-kubernetes\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971542 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cni-binary-copy\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971565 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-cni-netd\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971589 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb4nz\" (UniqueName: \"kubernetes.io/projected/f41571ca-a223-43b9-bed4-f4975da190db-kube-api-access-wb4nz\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971611 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76314433-6fa3-4bbe-835a-075283005f46-cni-binary-copy\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971651 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.972117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971678 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-kubernetes\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971705 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/291d4321-4e22-4921-a84c-707139c9b66a-kubelet-config\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971729 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-node-log\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971749 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-systemd\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971754 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-cni-multus\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971788 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-ovnkube-config\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971793 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-cni-multus\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971805 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-ovn\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971824 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971848 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971866 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysctl-conf\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971884 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971894 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-lib-modules\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971908 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-env-overrides\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971932 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-cni-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971946 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cnibin\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971945 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-slash\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.972907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971966 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-system-cni-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971998 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-cnibin\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972021 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-cni-bin\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972031 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-system-cni-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972043 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-modprobe-d\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972066 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-var-lib-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972074 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-k8s-cni-cncf-io\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972102 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972135 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-var-lib-kubelet\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972142 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-kubelet\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972168 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-kubelet\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972339 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972369 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-systemd-units\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972380 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-host\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972394 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-cni-bin\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972408 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-log-socket\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972420 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gvbk\" (UniqueName: \"kubernetes.io/projected/76314433-6fa3-4bbe-835a-075283005f46-kube-api-access-9gvbk\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972444 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-systemd\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.973769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972467 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-socket-dir-parent\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972507 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-hostroot\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972607 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-hostroot\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972651 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-run-ovn-kubernetes\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972691 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-cni-netd\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972704 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76314433-6fa3-4bbe-835a-075283005f46-multus-daemon-config\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972786 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972864 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/291d4321-4e22-4921-a84c-707139c9b66a-dbus\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.972906 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-sysctl-conf\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973327 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-env-overrides\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973394 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-cni-dir\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973433 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-cnibin\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973466 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-ovn\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973498 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76314433-6fa3-4bbe-835a-075283005f46-cni-binary-copy\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973524 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-ovnkube-config\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973518 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973555 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-run-netns\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973586 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-netns\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.974510 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973596 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-run-multus-certs\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973620 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-kubernetes\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.971707 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f41571ca-a223-43b9-bed4-f4975da190db-ovnkube-script-lib\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973690 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/291d4321-4e22-4921-a84c-707139c9b66a-kubelet-config\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973705 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-node-log\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973739 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-cnibin\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973744 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-systemd-units\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973758 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-kubelet\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973809 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-host-var-lib-cni-bin\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973840 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-host-cni-bin\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973840 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-modprobe-d\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973876 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-var-lib-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973937 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-multus-socket-dir-parent\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973978 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-systemd\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.973634 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f41571ca-a223-43b9-bed4-f4975da190db-run-openvswitch\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.974043 2538 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:40.974112 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret podName:291d4321-4e22-4921-a84c-707139c9b66a nodeName:}" failed. No retries permitted until 2026-09-09 20:47:41.474095734 +0000 UTC m=+3.092755959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret") pod "global-pull-secret-syncer-fd8jq" (UID: "291d4321-4e22-4921-a84c-707139c9b66a") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:40.975184 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.974158 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76314433-6fa3-4bbe-835a-075283005f46-etc-kubernetes\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:40.975861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.974164 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.975861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.974452 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f41571ca-a223-43b9-bed4-f4975da190db-ovn-node-metrics-cert\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.975861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.975071 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-tmp\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.976071 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.976054 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-etc-tuned\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.980579 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.980549 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ptvl\" (UniqueName: \"kubernetes.io/projected/fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040-kube-api-access-9ptvl\") pod \"multus-additional-cni-plugins-h64f5\" (UID: \"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040\") " pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:40.980795 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.980777 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79d5j\" (UniqueName: \"kubernetes.io/projected/a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788-kube-api-access-79d5j\") pod \"tuned-9m89j\" (UID: \"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788\") " pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:40.989563 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.989541 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb4nz\" (UniqueName: \"kubernetes.io/projected/f41571ca-a223-43b9-bed4-f4975da190db-kube-api-access-wb4nz\") pod \"ovnkube-node-t98tr\" (UID: \"f41571ca-a223-43b9-bed4-f4975da190db\") " pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:40.989563 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:40.989557 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gvbk\" (UniqueName: \"kubernetes.io/projected/76314433-6fa3-4bbe-835a-075283005f46-kube-api-access-9gvbk\") pod \"multus-rnlpl\" (UID: \"76314433-6fa3-4bbe-835a-075283005f46\") " pod="openshift-multus/multus-rnlpl" Sep 09 20:47:41.058548 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.058518 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mchtr" Sep 09 20:47:41.066566 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.066542 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wjgh2" Sep 09 20:47:41.077305 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.077284 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" Sep 09 20:47:41.084904 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.084882 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:47:41.091460 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.091440 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vgpth" Sep 09 20:47:41.097878 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.097859 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h64f5" Sep 09 20:47:41.106448 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.106423 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9m89j" Sep 09 20:47:41.112021 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.112002 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:47:41.117601 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.117582 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rnlpl" Sep 09 20:47:41.375753 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.375725 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:41.375902 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.375863 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:41.375938 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.375927 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:42.375905386 +0000 UTC m=+3.994565596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:41.477040 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.477010 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:41.477180 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.477060 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:41.477180 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.477163 2538 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:41.477307 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.477191 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:47:41.477307 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.477208 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:47:41.477307 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.477237 2538 projected.go:196] Error preparing data for projected volume kube-api-access-fbwmm for pod openshift-network-diagnostics/network-check-target-5kl9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:41.477307 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.477263 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret podName:291d4321-4e22-4921-a84c-707139c9b66a nodeName:}" failed. No retries permitted until 2026-09-09 20:47:42.477240025 +0000 UTC m=+4.095900251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret") pod "global-pull-secret-syncer-fd8jq" (UID: "291d4321-4e22-4921-a84c-707139c9b66a") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:41.477307 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.477287 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm podName:c6949849-ef14-4726-9cbf-6c5fce30374e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:42.47727132 +0000 UTC m=+4.095931558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwmm" (UniqueName: "kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm") pod "network-check-target-5kl9j" (UID: "c6949849-ef14-4726-9cbf-6c5fce30374e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:41.727593 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:41.727566 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76314433_6fa3_4bbe_835a_075283005f46.slice/crio-1b7534a6a64e67758a9ae6c26b482034fa9d494f12dad6f81074d28e1e6b5b66 WatchSource:0}: Error finding container 1b7534a6a64e67758a9ae6c26b482034fa9d494f12dad6f81074d28e1e6b5b66: Status 404 returned error can't find the container with id 1b7534a6a64e67758a9ae6c26b482034fa9d494f12dad6f81074d28e1e6b5b66 Sep 09 20:47:41.728681 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:41.728659 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b00b32_5bd9_4dbf_88e8_d5a2bcceb788.slice/crio-56015c0b338bb66cdc5b11bb469619c5ed9603f151f9aef852541b7ac17753f2 WatchSource:0}: Error finding container 56015c0b338bb66cdc5b11bb469619c5ed9603f151f9aef852541b7ac17753f2: Status 404 returned error can't find the container with id 56015c0b338bb66cdc5b11bb469619c5ed9603f151f9aef852541b7ac17753f2 Sep 09 20:47:41.729711 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:41.729690 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b239fd_3642_4f5d_9380_8c7823fd82ed.slice/crio-1ab8d0cdf2443b9b57de225200f1f165bc2b766dfccf543e5074f05e88f25b60 WatchSource:0}: Error finding container 1ab8d0cdf2443b9b57de225200f1f165bc2b766dfccf543e5074f05e88f25b60: Status 404 returned error can't find the container with id 1ab8d0cdf2443b9b57de225200f1f165bc2b766dfccf543e5074f05e88f25b60 Sep 09 20:47:41.733093 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:41.733074 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf41571ca_a223_43b9_bed4_f4975da190db.slice/crio-62b220e36db4c78c24b4ccc1f8869676ee86a07f2d0c9316714f9cd5e09a3393 WatchSource:0}: Error finding container 62b220e36db4c78c24b4ccc1f8869676ee86a07f2d0c9316714f9cd5e09a3393: Status 404 returned error can't find the container with id 62b220e36db4c78c24b4ccc1f8869676ee86a07f2d0c9316714f9cd5e09a3393 Sep 09 20:47:41.733750 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:41.733729 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0933970_91fd_4048_b522_eaba643670bb.slice/crio-f0fb839a5d3ad392bf7d35cf5cae8c4e0e5149376ca445f3a162cf24d1bc454a WatchSource:0}: Error finding container f0fb839a5d3ad392bf7d35cf5cae8c4e0e5149376ca445f3a162cf24d1bc454a: Status 404 returned error can't find the container with id f0fb839a5d3ad392bf7d35cf5cae8c4e0e5149376ca445f3a162cf24d1bc454a Sep 09 20:47:41.741505 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:47:41.741306 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a757be_03e0_47f8_9cd4_efa063d271a9.slice/crio-22f65200bb93c37f6104e2aa69635dc83836d76a8dee7726dac565a34e32a05d WatchSource:0}: Error finding container 22f65200bb93c37f6104e2aa69635dc83836d76a8dee7726dac565a34e32a05d: Status 404 returned error can't find the container with id 22f65200bb93c37f6104e2aa69635dc83836d76a8dee7726dac565a34e32a05d Sep 09 20:47:41.818717 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.818690 2538 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:42:40 +0000 UTC" deadline="2028-05-04 20:57:04.640851403 +0000 UTC" Sep 09 20:47:41.818717 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.818717 2538 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14472h9m22.822137652s" Sep 09 20:47:41.864437 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.864415 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:41.864535 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.864446 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:41.864602 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.864528 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:41.864669 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:41.864636 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:41.874803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.874782 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vgpth" event={"ID":"f0933970-91fd-4048-b522-eaba643670bb","Type":"ContainerStarted","Data":"f0fb839a5d3ad392bf7d35cf5cae8c4e0e5149376ca445f3a162cf24d1bc454a"} Sep 09 20:47:41.876729 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.876707 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" event={"ID":"80bed961383d1f2e370365643dbe1e93","Type":"ContainerStarted","Data":"baea51b50d01674cdc7d1f3118f9a32f397d73043bd7861aace77965a8312ff2"} Sep 09 20:47:41.878067 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.878045 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"62b220e36db4c78c24b4ccc1f8869676ee86a07f2d0c9316714f9cd5e09a3393"} Sep 09 20:47:41.879354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.879324 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dcdj7" event={"ID":"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa","Type":"ContainerStarted","Data":"6e26cc2be65231e7b4d66a867476c66b139bf153091016f3f872681020265845"} Sep 09 20:47:41.880381 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.880357 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wjgh2" event={"ID":"03b239fd-3642-4f5d-9380-8c7823fd82ed","Type":"ContainerStarted","Data":"1ab8d0cdf2443b9b57de225200f1f165bc2b766dfccf543e5074f05e88f25b60"} Sep 09 20:47:41.881522 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.881474 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9m89j" event={"ID":"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788","Type":"ContainerStarted","Data":"56015c0b338bb66cdc5b11bb469619c5ed9603f151f9aef852541b7ac17753f2"} Sep 09 20:47:41.882759 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.882731 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rnlpl" event={"ID":"76314433-6fa3-4bbe-835a-075283005f46","Type":"ContainerStarted","Data":"1b7534a6a64e67758a9ae6c26b482034fa9d494f12dad6f81074d28e1e6b5b66"} Sep 09 20:47:41.884149 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.884101 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerStarted","Data":"445f5eee0a0a5b7296f32084f151443cc58ac093049f0564c8d69447513e5909"} Sep 09 20:47:41.885277 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.885249 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mchtr" event={"ID":"62a757be-03e0-47f8-9cd4-efa063d271a9","Type":"ContainerStarted","Data":"22f65200bb93c37f6104e2aa69635dc83836d76a8dee7726dac565a34e32a05d"} Sep 09 20:47:41.886477 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:41.886461 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" event={"ID":"7cc5b2fa-35fe-47b7-836b-33e555c99943","Type":"ContainerStarted","Data":"e7c65cd8505ab8b65616c03666ccc2479305d9475b5a6ae6c25b048d9101f28e"} Sep 09 20:47:42.385215 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.385172 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:42.385393 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.385340 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:42.385441 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.385412 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:44.385394045 +0000 UTC m=+6.004054260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:42.486093 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.486055 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:42.486282 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.486122 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:42.486356 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.486336 2538 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:42.486736 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.486403 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret podName:291d4321-4e22-4921-a84c-707139c9b66a nodeName:}" failed. No retries permitted until 2026-09-09 20:47:44.486384366 +0000 UTC m=+6.105044593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret") pod "global-pull-secret-syncer-fd8jq" (UID: "291d4321-4e22-4921-a84c-707139c9b66a") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:42.487089 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.486973 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:47:42.487089 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.487003 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:47:42.487089 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.487015 2538 projected.go:196] Error preparing data for projected volume kube-api-access-fbwmm for pod openshift-network-diagnostics/network-check-target-5kl9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:42.487089 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.487071 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm podName:c6949849-ef14-4726-9cbf-6c5fce30374e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:44.487055104 +0000 UTC m=+6.105715317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwmm" (UniqueName: "kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm") pod "network-check-target-5kl9j" (UID: "c6949849-ef14-4726-9cbf-6c5fce30374e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:42.819883 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.819795 2538 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:42:40 +0000 UTC" deadline="2028-02-05 21:00:04.237080411 +0000 UTC" Sep 09 20:47:42.819883 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.819853 2538 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12336h12m21.417231449s" Sep 09 20:47:42.865248 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.864384 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:42.865248 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:42.864517 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:42.899861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.899043 2538 generic.go:356] "Generic (PLEG): container finished" podID="d66a9a646305fb874ccf05b3f9514ed5" containerID="44cc6e7d35e695b6a66470ef53478f2921312b756b1ca2a48c0b2cc8b48c046a" exitCode=0 Sep 09 20:47:42.899861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.899125 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"d66a9a646305fb874ccf05b3f9514ed5","Type":"ContainerDied","Data":"44cc6e7d35e695b6a66470ef53478f2921312b756b1ca2a48c0b2cc8b48c046a"} Sep 09 20:47:42.914662 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:42.914591 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" podStartSLOduration=2.914572246 podStartE2EDuration="2.914572246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:47:41.894752345 +0000 UTC m=+3.513412576" watchObservedRunningTime="2026-09-09 20:47:42.914572246 +0000 UTC m=+4.533232479" Sep 09 20:47:43.865245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:43.864988 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:43.865715 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:43.865347 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:43.865715 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:43.865039 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:43.865715 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:43.865681 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:43.931204 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:43.930478 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"d66a9a646305fb874ccf05b3f9514ed5","Type":"ContainerStarted","Data":"f842e5e0536d9c8fa57ae1ffd8fef03cd94a0962ac75edbd31ae8de5b5a374bb"} Sep 09 20:47:43.948591 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:43.947544 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" podStartSLOduration=3.947525213 podStartE2EDuration="3.947525213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:47:43.947275993 +0000 UTC m=+5.565936227" watchObservedRunningTime="2026-09-09 20:47:43.947525213 +0000 UTC m=+5.566185447" Sep 09 20:47:44.404123 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:44.404084 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:44.404321 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.404283 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:44.404386 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.404348 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:48.40432834 +0000 UTC m=+10.022988553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:44.504896 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:44.504854 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:44.505052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:44.504908 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:44.505052 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.505045 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:47:44.505176 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.505066 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:47:44.505176 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.505079 2538 projected.go:196] Error preparing data for projected volume kube-api-access-fbwmm for pod openshift-network-diagnostics/network-check-target-5kl9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:44.505176 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.505138 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm podName:c6949849-ef14-4726-9cbf-6c5fce30374e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:48.505118995 +0000 UTC m=+10.123779217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwmm" (UniqueName: "kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm") pod "network-check-target-5kl9j" (UID: "c6949849-ef14-4726-9cbf-6c5fce30374e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:44.505356 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.505214 2538 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:44.505356 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.505268 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret podName:291d4321-4e22-4921-a84c-707139c9b66a nodeName:}" failed. No retries permitted until 2026-09-09 20:47:48.505256348 +0000 UTC m=+10.123916564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret") pod "global-pull-secret-syncer-fd8jq" (UID: "291d4321-4e22-4921-a84c-707139c9b66a") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:44.869039 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:44.869007 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:44.869505 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.869140 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:44.869505 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:44.869488 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:44.869626 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:44.869579 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:45.865036 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:45.865001 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:45.865218 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:45.865123 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:46.864672 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:46.864634 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:46.865105 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:46.864791 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:46.865167 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:46.864638 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:46.865256 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:46.865217 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:47.865330 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:47.865294 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:47.865764 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:47.865427 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:48.435639 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:48.435592 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:48.435830 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.435760 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:48.435830 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.435828 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:56.435813068 +0000 UTC m=+18.054473279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:48.536448 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:48.536557 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.536652 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.536671 2538 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.536682 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.536695 2538 projected.go:196] Error preparing data for projected volume kube-api-access-fbwmm for pod openshift-network-diagnostics/network-check-target-5kl9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.536730 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret podName:291d4321-4e22-4921-a84c-707139c9b66a nodeName:}" failed. No retries permitted until 2026-09-09 20:47:56.536717355 +0000 UTC m=+18.155377564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret") pod "global-pull-secret-syncer-fd8jq" (UID: "291d4321-4e22-4921-a84c-707139c9b66a") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:48.537076 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.536746 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm podName:c6949849-ef14-4726-9cbf-6c5fce30374e nodeName:}" failed. No retries permitted until 2026-09-09 20:47:56.536740227 +0000 UTC m=+18.155400437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwmm" (UniqueName: "kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm") pod "network-check-target-5kl9j" (UID: "c6949849-ef14-4726-9cbf-6c5fce30374e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:48.865884 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:48.865847 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:48.866245 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.865956 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:48.866326 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:48.866304 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:48.866427 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:48.866406 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:49.865533 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:49.865071 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:49.865533 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:49.865194 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:50.864789 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:50.864750 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:50.865263 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:50.864805 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:50.865263 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:50.864894 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:50.866974 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:50.866943 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:51.864558 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:51.864516 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:51.864735 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:51.864644 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:52.865149 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:52.865111 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:52.865645 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:52.865151 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:52.865645 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:52.865331 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:52.865645 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:52.865466 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:53.865243 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:53.865195 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:53.865712 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:53.865357 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:54.865041 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:54.865007 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:54.865208 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:54.865117 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:54.865208 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:54.865155 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:54.865657 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:54.865237 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:55.864554 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:55.864524 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:55.864828 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:55.864641 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:56.496277 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:56.496210 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:56.496737 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.496361 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:56.496737 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.496428 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:48:12.496410459 +0000 UTC m=+34.115070674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:47:56.597055 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:56.597019 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:56.597055 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:56.597061 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:56.597283 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.597173 2538 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:56.597283 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.597183 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:47:56.597283 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.597197 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:47:56.597283 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.597209 2538 projected.go:196] Error preparing data for projected volume kube-api-access-fbwmm for pod openshift-network-diagnostics/network-check-target-5kl9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:56.597283 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.597258 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret podName:291d4321-4e22-4921-a84c-707139c9b66a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:12.597239949 +0000 UTC m=+34.215900178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret") pod "global-pull-secret-syncer-fd8jq" (UID: "291d4321-4e22-4921-a84c-707139c9b66a") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:47:56.597283 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.597273 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm podName:c6949849-ef14-4726-9cbf-6c5fce30374e nodeName:}" failed. No retries permitted until 2026-09-09 20:48:12.597266487 +0000 UTC m=+34.215926701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwmm" (UniqueName: "kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm") pod "network-check-target-5kl9j" (UID: "c6949849-ef14-4726-9cbf-6c5fce30374e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:47:56.865235 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:56.865193 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:56.865404 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.865335 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:56.865471 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:56.865394 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:56.865555 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:56.865537 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:57.864270 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:57.864234 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:57.864673 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:57.864364 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:47:58.865541 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:58.865505 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:47:58.865921 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:58.865620 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:47:58.865921 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:58.865653 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:47:58.865921 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:58.865746 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:47:59.864424 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:47:59.864369 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:47:59.864583 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:47:59.864482 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:48:00.864749 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.864541 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:00.865625 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.864609 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:00.865625 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:00.864845 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:48:00.865625 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:00.864953 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:48:00.982572 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.982527 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"ab1c593c70dadab07c0c33c13f89d16a16efeca5a208c47ad293bada0081d873"} Sep 09 20:48:00.982722 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.982630 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"04086daf397243733386395ea210f6836c5f67f710829d39dc30ad92b4c1bbb6"} Sep 09 20:48:00.982722 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.982676 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"87aa3043bf1cb61dc57200f7fc5884513f95a94a716fd89181fd8e1815d0113c"} Sep 09 20:48:00.982722 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.982704 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"17ef77fcfa0c9a871e510111b6820949b0027b0eb7f7a42587ab5dc30726657c"} Sep 09 20:48:00.982861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.982740 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"79c077a6ef2264cae10cf7065b8929f0029204d0039796d550f836ab879e3433"} Sep 09 20:48:00.982861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.982787 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"cc44e75fd98e9aabde050efa9ceb5cf588faed81a861771b14d4117f9dbc9ccc"} Sep 09 20:48:00.983866 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.983841 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dcdj7" event={"ID":"721b84f2-9419-4d85-bc3f-d3ed7f94d9aa","Type":"ContainerStarted","Data":"b633446f9edd60e0c23fd0ab6a2d90d3c80b71a67f05697b4d001278a4031926"} Sep 09 20:48:00.985380 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.985353 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wjgh2" event={"ID":"03b239fd-3642-4f5d-9380-8c7823fd82ed","Type":"ContainerStarted","Data":"d22b19572b92b3af1c8141bb0bdb65f2be89dfcf69b7b3d5091561d061756def"} Sep 09 20:48:00.986624 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.986605 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9m89j" event={"ID":"a7b00b32-5bd9-4dbf-88e8-d5a2bcceb788","Type":"ContainerStarted","Data":"de00a3ae116f549a130d41ecf9601696777fd526e19535ecc5ea232d1721eb5d"} Sep 09 20:48:00.988104 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.988086 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rnlpl" event={"ID":"76314433-6fa3-4bbe-835a-075283005f46","Type":"ContainerStarted","Data":"cf8983da6c46a6f9112fe98c19c3144fbc45959e11ff5f14f98e93ed0137822e"} Sep 09 20:48:00.989523 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.989503 2538 generic.go:356] "Generic (PLEG): container finished" podID="fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040" containerID="f3370d95716cf69e00c484321cf50e1c93f4116c027ac3b209ecc23e4a88cb1d" exitCode=0 Sep 09 20:48:00.989692 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.989560 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerDied","Data":"f3370d95716cf69e00c484321cf50e1c93f4116c027ac3b209ecc23e4a88cb1d"} Sep 09 20:48:00.991033 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.991014 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" event={"ID":"7cc5b2fa-35fe-47b7-836b-33e555c99943","Type":"ContainerStarted","Data":"2fac1bd7b26b06d0febe3587bfa789da45e07d0a12aae6a124598348c194c52a"} Sep 09 20:48:00.992333 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:00.992314 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vgpth" event={"ID":"f0933970-91fd-4048-b522-eaba643670bb","Type":"ContainerStarted","Data":"bca7039955c1c8ad7207efb955dc3fe7f6052be593954d6825324b76a37d31cd"} Sep 09 20:48:01.008797 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.008752 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vgpth" podStartSLOduration=3.88214902 podStartE2EDuration="22.008739614s" totalImagesPullingTime="18.126590594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:01.008717042 +0000 UTC m=+22.627377297" watchObservedRunningTime="2026-09-09 20:48:01.008739614 +0000 UTC m=+22.627399846" Sep 09 20:48:01.009131 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.009111 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dcdj7" podStartSLOduration=8.011720464 podStartE2EDuration="22.009104567s" totalImagesPullingTime="13.997384103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:00.996496716 +0000 UTC m=+22.615156947" watchObservedRunningTime="2026-09-09 20:48:01.009104567 +0000 UTC m=+22.627764810" Sep 09 20:48:01.047837 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.047791 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9m89j" podStartSLOduration=3.908645815 podStartE2EDuration="22.047775564s" totalImagesPullingTime="18.139129749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:01.047245758 +0000 UTC m=+22.665905987" watchObservedRunningTime="2026-09-09 20:48:01.047775564 +0000 UTC m=+22.666435814" Sep 09 20:48:01.064308 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.064257 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rnlpl" podStartSLOduration=3.871348026 podStartE2EDuration="22.064243045s" totalImagesPullingTime="18.192895019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:01.063669476 +0000 UTC m=+22.682329707" watchObservedRunningTime="2026-09-09 20:48:01.064243045 +0000 UTC m=+22.682903268" Sep 09 20:48:01.081934 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.081885 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wjgh2" podStartSLOduration=4.944866091 podStartE2EDuration="23.081868798s" totalImagesPullingTime="18.137002707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:01.081451088 +0000 UTC m=+22.700111320" watchObservedRunningTime="2026-09-09 20:48:01.081868798 +0000 UTC m=+22.700529028" Sep 09 20:48:01.093929 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.093897 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:01.257476 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.257345 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:01.285376 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.285353 2538 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 20:48:01.359068 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.359042 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:01.840618 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.840501 2538 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T20:48:01.285370897Z","UUID":"4408ff69-10a7-494c-893b-45f3f676f40f","Handler":null,"Name":"","Endpoint":""} Sep 09 20:48:01.842195 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.842170 2538 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 20:48:01.842342 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.842205 2538 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 20:48:01.864591 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.864564 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:01.864708 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:01.864692 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:48:01.997385 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.997348 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mchtr" event={"ID":"62a757be-03e0-47f8-9cd4-efa063d271a9","Type":"ContainerStarted","Data":"493255ee9c2b3c38c2e65d2efda8b98d24e0230249b293f9cae1e75b5a2cb417"} Sep 09 20:48:01.999438 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:01.999411 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" event={"ID":"7cc5b2fa-35fe-47b7-836b-33e555c99943","Type":"ContainerStarted","Data":"44a83d68d4da5f0f59cd39737fdfd52608c535e9965c23b02078103c97b4c4ed"} Sep 09 20:48:02.013676 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:02.013635 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mchtr" podStartSLOduration=5.892185777 podStartE2EDuration="24.013621968s" totalImagesPullingTime="18.121436191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:02.012483195 +0000 UTC m=+23.631143425" watchObservedRunningTime="2026-09-09 20:48:02.013621968 +0000 UTC m=+23.632282199" Sep 09 20:48:02.131806 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:02.131707 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:02.201590 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:02.201556 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:02.865104 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:02.865075 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:02.865298 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:02.865203 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:48:02.865298 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:02.865256 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:02.865427 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:02.865378 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:48:03.006717 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:03.006641 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"69e959a6b748717b4bfbb40f065aac2ce886d3a7f22543bf167f7965233005cf"} Sep 09 20:48:03.009328 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:03.009295 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" event={"ID":"7cc5b2fa-35fe-47b7-836b-33e555c99943","Type":"ContainerStarted","Data":"485fb46552cf7d55c8a0907c292a4c7774962aafe17e808b53c926e005ac0d2e"} Sep 09 20:48:03.027617 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:03.027575 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gw4qf" podStartSLOduration=4.754981875 podStartE2EDuration="25.027557799s" totalImagesPullingTime="20.272575924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:03.027165211 +0000 UTC m=+24.645825447" watchObservedRunningTime="2026-09-09 20:48:03.027557799 +0000 UTC m=+24.646218032" Sep 09 20:48:03.393978 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:03.393937 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:03.449349 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:03.449316 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:03.865205 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:03.865173 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:03.865398 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:03.865294 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:48:04.378538 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:04.378503 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:48:04.379418 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:04.379120 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:48:04.864516 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:04.864485 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:04.864672 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:04.864490 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:04.864672 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:04.864624 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:48:04.864766 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:04.864697 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:48:05.864582 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:05.864378 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:05.865139 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:05.864620 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:48:06.021624 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.021585 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" event={"ID":"f41571ca-a223-43b9-bed4-f4975da190db","Type":"ContainerStarted","Data":"a1505d5221fe00aaf33763b14416de884f773a428517719a48cebb41fcd7cced"} Sep 09 20:48:06.021889 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.021868 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:48:06.021994 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.021939 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:48:06.023705 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.023678 2538 generic.go:356] "Generic (PLEG): container finished" podID="fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040" containerID="afa93f2c79da32e3204e65066439fa232dd87a788dd7ee460cf4d4ead0eb1c40" exitCode=0 Sep 09 20:48:06.023797 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.023727 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerDied","Data":"afa93f2c79da32e3204e65066439fa232dd87a788dd7ee460cf4d4ead0eb1c40"} Sep 09 20:48:06.038099 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.038076 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:48:06.050281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.050245 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" podStartSLOduration=8.853337002 podStartE2EDuration="27.050215758s" totalImagesPullingTime="18.196878756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:06.049711692 +0000 UTC m=+27.668371934" watchObservedRunningTime="2026-09-09 20:48:06.050215758 +0000 UTC m=+27.668875989" Sep 09 20:48:06.865243 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.865196 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:06.865809 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.865196 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:06.865809 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:06.865355 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:48:06.865809 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:06.865421 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:48:06.927145 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.927117 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:48:06.927303 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.927264 2538 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 20:48:06.927740 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.927715 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dcdj7" Sep 09 20:48:06.991338 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:06.991310 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:07.025802 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.025768 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:48:07.042276 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.042249 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:48:07.305798 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.304842 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fd8jq"] Sep 09 20:48:07.305798 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.305671 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-84ccl"] Sep 09 20:48:07.305798 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.305719 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:07.306130 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.306109 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:07.306279 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:07.306253 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:48:07.307518 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:07.307048 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:48:07.307981 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.307691 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5kl9j"] Sep 09 20:48:07.307981 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:07.307781 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:07.307981 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:07.307885 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:48:08.029840 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:08.029810 2538 generic.go:356] "Generic (PLEG): container finished" podID="fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040" containerID="c57c26643f5a71c0752c8eb55eaa503903db2ade5d23e80edf85b4c3119f9c9f" exitCode=0 Sep 09 20:48:08.030481 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:08.029903 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerDied","Data":"c57c26643f5a71c0752c8eb55eaa503903db2ade5d23e80edf85b4c3119f9c9f"} Sep 09 20:48:08.865246 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:08.865198 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:08.865434 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:08.865302 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:48:08.865434 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:08.865322 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:08.865434 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:08.865336 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:08.865434 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:08.865421 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:48:08.865635 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:08.865510 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:48:10.036627 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:10.036590 2538 generic.go:356] "Generic (PLEG): container finished" podID="fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040" containerID="53265219db934b0139ca0c6694768d3d10747eb8205c7dfea833c9a2eeb3b77f" exitCode=0 Sep 09 20:48:10.037002 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:10.036635 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerDied","Data":"53265219db934b0139ca0c6694768d3d10747eb8205c7dfea833c9a2eeb3b77f"} Sep 09 20:48:10.864731 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:10.864691 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:10.864884 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:10.864810 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:48:10.864884 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:10.864691 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:10.864884 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:10.864873 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fd8jq" podUID="291d4321-4e22-4921-a84c-707139c9b66a" Sep 09 20:48:10.864884 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:10.864691 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:10.865031 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:10.864926 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5kl9j" podUID="c6949849-ef14-4726-9cbf-6c5fce30374e" Sep 09 20:48:12.512647 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.512611 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:12.513032 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.512751 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:48:12.513032 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.512825 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:48:44.512804154 +0000 UTC m=+66.131464369 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:48:12.613124 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.613081 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:12.613294 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.613141 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:12.613294 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.613265 2538 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:48:12.613294 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.613279 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:48:12.613294 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.613296 2538 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:48:12.613458 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.613310 2538 projected.go:196] Error preparing data for projected volume kube-api-access-fbwmm for pod openshift-network-diagnostics/network-check-target-5kl9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:48:12.613458 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.613338 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret podName:291d4321-4e22-4921-a84c-707139c9b66a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:44.613318044 +0000 UTC m=+66.231978258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret") pod "global-pull-secret-syncer-fd8jq" (UID: "291d4321-4e22-4921-a84c-707139c9b66a") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:48:12.613458 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:12.613357 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm podName:c6949849-ef14-4726-9cbf-6c5fce30374e nodeName:}" failed. No retries permitted until 2026-09-09 20:48:44.613348039 +0000 UTC m=+66.232008254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwmm" (UniqueName: "kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm") pod "network-check-target-5kl9j" (UID: "c6949849-ef14-4726-9cbf-6c5fce30374e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:48:12.712770 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.712696 2538 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeReady" Sep 09 20:48:12.712910 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.712843 2538 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 20:48:12.758729 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.758697 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m2r86"] Sep 09 20:48:12.760708 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.760682 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:12.762948 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.762925 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-49xnp"] Sep 09 20:48:12.763426 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.763401 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 20:48:12.763612 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.763593 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5nprp\"" Sep 09 20:48:12.764289 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.764269 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 20:48:12.764760 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.764745 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:12.767255 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.767216 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 20:48:12.767342 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.767272 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 20:48:12.768092 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.768048 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 20:48:12.768283 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.768259 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bv8pc\"" Sep 09 20:48:12.774890 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.774865 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m2r86"] Sep 09 20:48:12.778823 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.778801 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-49xnp"] Sep 09 20:48:12.864585 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.864554 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:12.864735 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.864557 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:12.864735 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.864670 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:12.867763 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.867602 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-slrrp\"" Sep 09 20:48:12.867763 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.867633 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kzsmd\"" Sep 09 20:48:12.867763 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.867612 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 20:48:12.867763 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.867599 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 20:48:12.867763 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.867669 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 20:48:12.868094 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.868035 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 20:48:12.911424 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.911392 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:12.915791 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.915763 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2bpk\" (UniqueName: \"kubernetes.io/projected/ef7c014b-1fab-4d97-8831-095a7712283b-kube-api-access-f2bpk\") pod \"ingress-canary-49xnp\" (UID: \"ef7c014b-1fab-4d97-8831-095a7712283b\") " pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:12.915791 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.915793 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:12.915986 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.915813 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c681c3-1804-4094-bb89-687d2fdb9b8a-tmp-dir\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:12.915986 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.915886 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88c681c3-1804-4094-bb89-687d2fdb9b8a-config-volume\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:12.915986 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.915938 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq754\" (UniqueName: \"kubernetes.io/projected/88c681c3-1804-4094-bb89-687d2fdb9b8a-kube-api-access-kq754\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:12.915986 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.915970 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef7c014b-1fab-4d97-8831-095a7712283b-cert\") pod \"ingress-canary-49xnp\" (UID: \"ef7c014b-1fab-4d97-8831-095a7712283b\") " pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:12.966959 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:12.966875 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:13.016955 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.016915 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88c681c3-1804-4094-bb89-687d2fdb9b8a-config-volume\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.016955 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.016959 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq754\" (UniqueName: \"kubernetes.io/projected/88c681c3-1804-4094-bb89-687d2fdb9b8a-kube-api-access-kq754\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.017197 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.016988 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef7c014b-1fab-4d97-8831-095a7712283b-cert\") pod \"ingress-canary-49xnp\" (UID: \"ef7c014b-1fab-4d97-8831-095a7712283b\") " pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:13.017197 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.017061 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2bpk\" (UniqueName: \"kubernetes.io/projected/ef7c014b-1fab-4d97-8831-095a7712283b-kube-api-access-f2bpk\") pod \"ingress-canary-49xnp\" (UID: \"ef7c014b-1fab-4d97-8831-095a7712283b\") " pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:13.017197 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.017090 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.017197 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.017118 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c681c3-1804-4094-bb89-687d2fdb9b8a-tmp-dir\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.017430 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:13.017260 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:48:13.017430 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:13.017343 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:13.517322558 +0000 UTC m=+35.135982785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:48:13.017544 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.017451 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c681c3-1804-4094-bb89-687d2fdb9b8a-tmp-dir\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.017634 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.017614 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88c681c3-1804-4094-bb89-687d2fdb9b8a-config-volume\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.021928 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.021800 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef7c014b-1fab-4d97-8831-095a7712283b-cert\") pod \"ingress-canary-49xnp\" (UID: \"ef7c014b-1fab-4d97-8831-095a7712283b\") " pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:13.028553 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.028532 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq754\" (UniqueName: \"kubernetes.io/projected/88c681c3-1804-4094-bb89-687d2fdb9b8a-kube-api-access-kq754\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.028720 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.028694 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2bpk\" (UniqueName: \"kubernetes.io/projected/ef7c014b-1fab-4d97-8831-095a7712283b-kube-api-access-f2bpk\") pod \"ingress-canary-49xnp\" (UID: \"ef7c014b-1fab-4d97-8831-095a7712283b\") " pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:13.085333 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.085292 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-49xnp" Sep 09 20:48:13.520743 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:13.520707 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:13.521310 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:13.520901 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:48:13.521310 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:13.520980 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:14.520959608 +0000 UTC m=+36.139619822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:48:14.527564 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:14.527540 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:14.528348 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:14.527696 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:48:14.528348 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:14.527766 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:16.527741879 +0000 UTC m=+38.146402089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:48:14.665785 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:14.665712 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-49xnp"] Sep 09 20:48:14.669196 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:48:14.669173 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7c014b_1fab_4d97_8831_095a7712283b.slice/crio-36865cde743937420099fa5b2b689a96e7d78c9b1ee3771f160a60e2bf7f79c4 WatchSource:0}: Error finding container 36865cde743937420099fa5b2b689a96e7d78c9b1ee3771f160a60e2bf7f79c4: Status 404 returned error can't find the container with id 36865cde743937420099fa5b2b689a96e7d78c9b1ee3771f160a60e2bf7f79c4 Sep 09 20:48:15.051350 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:15.051320 2538 generic.go:356] "Generic (PLEG): container finished" podID="fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040" containerID="d84a60e54c38c932c007b7b6279d42a6bf89ec952b5e32b1a9931de5c6bb78ea" exitCode=0 Sep 09 20:48:15.051535 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:15.051409 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerDied","Data":"d84a60e54c38c932c007b7b6279d42a6bf89ec952b5e32b1a9931de5c6bb78ea"} Sep 09 20:48:15.052536 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:15.052506 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-49xnp" event={"ID":"ef7c014b-1fab-4d97-8831-095a7712283b","Type":"ContainerStarted","Data":"36865cde743937420099fa5b2b689a96e7d78c9b1ee3771f160a60e2bf7f79c4"} Sep 09 20:48:16.057586 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:16.057556 2538 generic.go:356] "Generic (PLEG): container finished" podID="fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040" containerID="10b763c2b0a8bd86340c38feeac0ea42b82314f54080df83d9b3b39fac13ba06" exitCode=0 Sep 09 20:48:16.058087 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:16.057614 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerDied","Data":"10b763c2b0a8bd86340c38feeac0ea42b82314f54080df83d9b3b39fac13ba06"} Sep 09 20:48:16.196803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:16.196766 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:16.258561 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:16.258521 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:16.440203 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:16.440122 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:16.539779 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:16.539751 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:16.542817 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:16.542789 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:16.542957 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:16.542943 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:48:16.543018 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:16.543008 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:20.542991566 +0000 UTC m=+42.161651782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:48:17.063444 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:17.063270 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h64f5" event={"ID":"fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040","Type":"ContainerStarted","Data":"f0bf0f0e0480e5a46509f167163cbea367f2b0fd28e67055eda4fcad7787131a"} Sep 09 20:48:17.087530 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:17.087479 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h64f5" podStartSLOduration=8.623303561 podStartE2EDuration="38.087465088s" totalImagesPullingTime="29.464161527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:17.085954982 +0000 UTC m=+38.704615215" watchObservedRunningTime="2026-09-09 20:48:17.087465088 +0000 UTC m=+38.706125319" Sep 09 20:48:17.207466 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:17.207442 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:17.309140 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:17.309107 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:17.553966 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:17.553938 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:17.653941 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:17.653909 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:18.068072 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:18.068040 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-49xnp" event={"ID":"ef7c014b-1fab-4d97-8831-095a7712283b","Type":"ContainerStarted","Data":"3f83ffc10afe08b816bc3657354522c95ce5778a959ed5b8578b597e21172a0d"} Sep 09 20:48:18.085305 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:18.085260 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-49xnp" podStartSLOduration=3.677031904 podStartE2EDuration="6.085243953s" totalImagesPullingTime="2.408212049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:18.084980666 +0000 UTC m=+39.703640888" watchObservedRunningTime="2026-09-09 20:48:18.085243953 +0000 UTC m=+39.703904176" Sep 09 20:48:18.086982 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:18.086959 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:20.571578 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:20.571528 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:20.572055 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:20.571700 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:48:20.572055 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:20.571781 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:28.571760928 +0000 UTC m=+50.190421155 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:48:28.623630 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:28.623591 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:28.623996 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:28.623702 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:48:28.623996 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:28.623751 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:48:44.62373875 +0000 UTC m=+66.242398964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:48:39.043084 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:39.043058 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t98tr" Sep 09 20:48:39.171582 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:39.171551 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:39.236715 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:39.236688 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:48:44.533125 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.533083 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:48:44.535952 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.535932 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 20:48:44.544278 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:44.544260 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 20:48:44.544360 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:44.544317 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:49:48.544301796 +0000 UTC m=+130.162962006 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : secret "metrics-daemon-secret" not found Sep 09 20:48:44.633610 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.633582 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:48:44.633728 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.633616 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:44.633728 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.633639 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:44.633843 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:44.633744 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:48:44.633843 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:48:44.633818 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:49:16.63378907 +0000 UTC m=+98.252449294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:48:44.636515 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.636495 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 20:48:44.636633 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.636617 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 20:48:44.648085 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.648063 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/291d4321-4e22-4921-a84c-707139c9b66a-original-pull-secret\") pod \"global-pull-secret-syncer-fd8jq\" (UID: \"291d4321-4e22-4921-a84c-707139c9b66a\") " pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:44.648411 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.648397 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 20:48:44.657591 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.657568 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbwmm\" (UniqueName: \"kubernetes.io/projected/c6949849-ef14-4726-9cbf-6c5fce30374e-kube-api-access-fbwmm\") pod \"network-check-target-5kl9j\" (UID: \"c6949849-ef14-4726-9cbf-6c5fce30374e\") " pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:44.678847 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.678827 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kzsmd\"" Sep 09 20:48:44.687286 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.687267 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:44.688439 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.688427 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fd8jq" Sep 09 20:48:44.988795 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:44.988764 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5kl9j"] Sep 09 20:48:44.992730 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:48:44.992706 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6949849_ef14_4726_9cbf_6c5fce30374e.slice/crio-d90ef02a962d5dde4ac8f349d3f4f19d2549a09789f88fc918751039a0b29936 WatchSource:0}: Error finding container d90ef02a962d5dde4ac8f349d3f4f19d2549a09789f88fc918751039a0b29936: Status 404 returned error can't find the container with id d90ef02a962d5dde4ac8f349d3f4f19d2549a09789f88fc918751039a0b29936 Sep 09 20:48:45.017310 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:45.017285 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fd8jq"] Sep 09 20:48:45.019672 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:48:45.019649 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291d4321_4e22_4921_a84c_707139c9b66a.slice/crio-a17cdc08945de8e1284ef89b960362d72e9a78a094bbc7d2879e2f808b84e15a WatchSource:0}: Error finding container a17cdc08945de8e1284ef89b960362d72e9a78a094bbc7d2879e2f808b84e15a: Status 404 returned error can't find the container with id a17cdc08945de8e1284ef89b960362d72e9a78a094bbc7d2879e2f808b84e15a Sep 09 20:48:45.131585 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:45.131553 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fd8jq" event={"ID":"291d4321-4e22-4921-a84c-707139c9b66a","Type":"ContainerStarted","Data":"a17cdc08945de8e1284ef89b960362d72e9a78a094bbc7d2879e2f808b84e15a"} Sep 09 20:48:45.132661 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:45.132637 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5kl9j" event={"ID":"c6949849-ef14-4726-9cbf-6c5fce30374e","Type":"ContainerStarted","Data":"d90ef02a962d5dde4ac8f349d3f4f19d2549a09789f88fc918751039a0b29936"} Sep 09 20:48:48.285464 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.285431 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw"] Sep 09 20:48:48.287484 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.287454 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.290150 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.290123 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 20:48:48.290293 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.290145 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 20:48:48.294052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.293855 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 20:48:48.294052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.293960 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 09 20:48:48.298004 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.297979 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw"] Sep 09 20:48:48.318721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.318679 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf"] Sep 09 20:48:48.320974 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.320953 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:48.323686 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.323659 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-rn4lc\"" Sep 09 20:48:48.323775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.323689 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 09 20:48:48.336380 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.336354 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf"] Sep 09 20:48:48.362265 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.362241 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59159d27-10f4-4781-9254-0c9c18bc85e3-tmp\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.362391 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.362279 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgxp4\" (UniqueName: \"kubernetes.io/projected/59159d27-10f4-4781-9254-0c9c18bc85e3-kube-api-access-qgxp4\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.362391 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.362369 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prks2\" (UniqueName: \"kubernetes.io/projected/08565fed-de99-4ebc-a1d0-27b69ad0e017-kube-api-access-prks2\") pod \"managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf\" (UID: \"08565fed-de99-4ebc-a1d0-27b69ad0e017\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:48.362516 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.362399 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08565fed-de99-4ebc-a1d0-27b69ad0e017-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf\" (UID: \"08565fed-de99-4ebc-a1d0-27b69ad0e017\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:48.362516 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.362425 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/59159d27-10f4-4781-9254-0c9c18bc85e3-klusterlet-config\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.463539 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.463507 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59159d27-10f4-4781-9254-0c9c18bc85e3-tmp\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.463539 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.463541 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgxp4\" (UniqueName: \"kubernetes.io/projected/59159d27-10f4-4781-9254-0c9c18bc85e3-kube-api-access-qgxp4\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.463774 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.463689 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prks2\" (UniqueName: \"kubernetes.io/projected/08565fed-de99-4ebc-a1d0-27b69ad0e017-kube-api-access-prks2\") pod \"managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf\" (UID: \"08565fed-de99-4ebc-a1d0-27b69ad0e017\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:48.463774 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.463737 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08565fed-de99-4ebc-a1d0-27b69ad0e017-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf\" (UID: \"08565fed-de99-4ebc-a1d0-27b69ad0e017\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:48.463774 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.463765 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/59159d27-10f4-4781-9254-0c9c18bc85e3-klusterlet-config\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.463933 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.463915 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59159d27-10f4-4781-9254-0c9c18bc85e3-tmp\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.466269 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.466250 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08565fed-de99-4ebc-a1d0-27b69ad0e017-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf\" (UID: \"08565fed-de99-4ebc-a1d0-27b69ad0e017\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:48.466569 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.466549 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/59159d27-10f4-4781-9254-0c9c18bc85e3-klusterlet-config\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.473448 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.473421 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prks2\" (UniqueName: \"kubernetes.io/projected/08565fed-de99-4ebc-a1d0-27b69ad0e017-kube-api-access-prks2\") pod \"managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf\" (UID: \"08565fed-de99-4ebc-a1d0-27b69ad0e017\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:48.473653 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.473628 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgxp4\" (UniqueName: \"kubernetes.io/projected/59159d27-10f4-4781-9254-0c9c18bc85e3-kube-api-access-qgxp4\") pod \"klusterlet-addon-workmgr-578597dd4f-55rtw\" (UID: \"59159d27-10f4-4781-9254-0c9c18bc85e3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.602183 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.602147 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:48.643239 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:48.643203 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" Sep 09 20:48:50.130848 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.130723 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf"] Sep 09 20:48:50.131443 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.130966 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw"] Sep 09 20:48:50.133837 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:48:50.133760 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08565fed_de99_4ebc_a1d0_27b69ad0e017.slice/crio-c8f68bdd607da5322772cb7d2f517c156c3b8b88423ed9559f4a092842005db0 WatchSource:0}: Error finding container c8f68bdd607da5322772cb7d2f517c156c3b8b88423ed9559f4a092842005db0: Status 404 returned error can't find the container with id c8f68bdd607da5322772cb7d2f517c156c3b8b88423ed9559f4a092842005db0 Sep 09 20:48:50.135586 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:48:50.135560 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59159d27_10f4_4781_9254_0c9c18bc85e3.slice/crio-41e065765b0c356571650732c432306af5e6692a72380744b0d391b2367f5290 WatchSource:0}: Error finding container 41e065765b0c356571650732c432306af5e6692a72380744b0d391b2367f5290: Status 404 returned error can't find the container with id 41e065765b0c356571650732c432306af5e6692a72380744b0d391b2367f5290 Sep 09 20:48:50.146527 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.146501 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5kl9j" event={"ID":"c6949849-ef14-4726-9cbf-6c5fce30374e","Type":"ContainerStarted","Data":"f478d08a8463000fea64f85cb21f69a993ce1c700f569a6f4cce900df53baad6"} Sep 09 20:48:50.146643 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.146585 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:48:50.148125 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.148097 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fd8jq" event={"ID":"291d4321-4e22-4921-a84c-707139c9b66a","Type":"ContainerStarted","Data":"465e736cc0bb550f4227ba8378ce3809b82cfa806f4eb678df3b2bb3f93af617"} Sep 09 20:48:50.149393 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.149374 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" event={"ID":"59159d27-10f4-4781-9254-0c9c18bc85e3","Type":"ContainerStarted","Data":"41e065765b0c356571650732c432306af5e6692a72380744b0d391b2367f5290"} Sep 09 20:48:50.150666 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.150645 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" event={"ID":"08565fed-de99-4ebc-a1d0-27b69ad0e017","Type":"ContainerStarted","Data":"c8f68bdd607da5322772cb7d2f517c156c3b8b88423ed9559f4a092842005db0"} Sep 09 20:48:50.162986 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.162945 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5kl9j" podStartSLOduration=67.29890612 podStartE2EDuration="1m12.16293269s" totalImagesPullingTime="4.86402657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:50.162335062 +0000 UTC m=+71.780995295" watchObservedRunningTime="2026-09-09 20:48:50.16293269 +0000 UTC m=+71.781592935" Sep 09 20:48:50.177391 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:50.177349 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fd8jq" podStartSLOduration=66.325602032 podStartE2EDuration="1m11.177340519s" totalImagesPullingTime="4.851738487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:50.176647051 +0000 UTC m=+71.795307282" watchObservedRunningTime="2026-09-09 20:48:50.177340519 +0000 UTC m=+71.796000752" Sep 09 20:48:56.168910 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:56.168875 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" event={"ID":"08565fed-de99-4ebc-a1d0-27b69ad0e017","Type":"ContainerStarted","Data":"900497ac58add5faf30ae26e5fb9e1a2b150a2445f692f9136bd022fdfa26b16"} Sep 09 20:48:56.170461 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:56.170438 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" event={"ID":"59159d27-10f4-4781-9254-0c9c18bc85e3","Type":"ContainerStarted","Data":"7299e9fe74d6f2488edaa2fc5b2190175bc0d351b65f8c3e30fe9e8b20a8a473"} Sep 09 20:48:56.170659 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:56.170643 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:56.172519 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:56.172496 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:48:56.183933 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:56.183896 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" podStartSLOduration=3.160207741 podStartE2EDuration="8.183883034s" totalImagesPullingTime="5.023675293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:56.183727026 +0000 UTC m=+77.802387259" watchObservedRunningTime="2026-09-09 20:48:56.183883034 +0000 UTC m=+77.802543260" Sep 09 20:48:56.200756 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:48:56.200676 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" podStartSLOduration=3.168483682 podStartE2EDuration="8.200659934s" totalImagesPullingTime="5.032176252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:56.200338036 +0000 UTC m=+77.818998268" watchObservedRunningTime="2026-09-09 20:48:56.200659934 +0000 UTC m=+77.819320167" Sep 09 20:49:16.654314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:49:16.654145 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:49:16.654314 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:49:16.654296 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:49:16.654741 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:49:16.654388 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:50:20.654373941 +0000 UTC m=+162.273034150 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:49:18.109178 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:49:18.109146 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:49:21.156443 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:49:21.156411 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5kl9j" Sep 09 20:49:48.563216 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:49:48.563179 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:49:48.563715 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:49:48.563341 2538 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 20:49:48.563715 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:49:48.563419 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs podName:b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:50.563398366 +0000 UTC m=+252.182058576 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs") pod "network-metrics-daemon-84ccl" (UID: "b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e") : secret "metrics-daemon-secret" not found Sep 09 20:50:15.776131 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:50:15.776085 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-m2r86" podUID="88c681c3-1804-4094-bb89-687d2fdb9b8a" Sep 09 20:50:15.883383 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:50:15.883350 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-84ccl" podUID="b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e" Sep 09 20:50:16.369414 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:16.369386 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m2r86" Sep 09 20:50:20.680063 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:20.680013 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:50:20.680482 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:50:20.680137 2538 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:50:20.680482 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:50:20.680190 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls podName:88c681c3-1804-4094-bb89-687d2fdb9b8a nodeName:}" failed. No retries permitted until 2026-09-09 20:52:22.680176494 +0000 UTC m=+284.298836717 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls") pod "dns-default-m2r86" (UID: "88c681c3-1804-4094-bb89-687d2fdb9b8a") : secret "dns-default-metrics-tls" not found Sep 09 20:50:26.864690 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:26.864657 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:50:55.463381 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:55.463354 2538 generic.go:356] "Generic (PLEG): container finished" podID="59159d27-10f4-4781-9254-0c9c18bc85e3" containerID="7299e9fe74d6f2488edaa2fc5b2190175bc0d351b65f8c3e30fe9e8b20a8a473" exitCode=1 Sep 09 20:50:55.463814 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:55.463429 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" event={"ID":"59159d27-10f4-4781-9254-0c9c18bc85e3","Type":"ContainerDied","Data":"7299e9fe74d6f2488edaa2fc5b2190175bc0d351b65f8c3e30fe9e8b20a8a473"} Sep 09 20:50:55.463814 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:55.463738 2538 scope.go:117] "RemoveContainer" containerID="7299e9fe74d6f2488edaa2fc5b2190175bc0d351b65f8c3e30fe9e8b20a8a473" Sep 09 20:50:55.465113 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:55.465092 2538 generic.go:356] "Generic (PLEG): container finished" podID="08565fed-de99-4ebc-a1d0-27b69ad0e017" containerID="900497ac58add5faf30ae26e5fb9e1a2b150a2445f692f9136bd022fdfa26b16" exitCode=255 Sep 09 20:50:55.465218 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:55.465158 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" event={"ID":"08565fed-de99-4ebc-a1d0-27b69ad0e017","Type":"ContainerDied","Data":"900497ac58add5faf30ae26e5fb9e1a2b150a2445f692f9136bd022fdfa26b16"} Sep 09 20:50:55.470751 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:55.470735 2538 scope.go:117] "RemoveContainer" containerID="900497ac58add5faf30ae26e5fb9e1a2b150a2445f692f9136bd022fdfa26b16" Sep 09 20:50:56.171579 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:56.171548 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:50:56.469698 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:56.469615 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" event={"ID":"59159d27-10f4-4781-9254-0c9c18bc85e3","Type":"ContainerStarted","Data":"eb44cfc128f685d16d30c077eb2cff03f4f510bdacd6b96b72626afcad9ba24f"} Sep 09 20:50:56.470115 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:56.469799 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:50:56.470460 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:56.470438 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-578597dd4f-55rtw" Sep 09 20:50:56.471489 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:50:56.471472 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d47d7f76b-vvtdf" event={"ID":"08565fed-de99-4ebc-a1d0-27b69ad0e017","Type":"ContainerStarted","Data":"c68d49042a61f7b12fde72d29bed660e299c9a008bd7bd2ef7c31e07cafd73d1"} Sep 09 20:51:32.415803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.415761 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9d949964-m9n92"] Sep 09 20:51:32.417749 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.417726 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.420326 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.420309 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 09 20:51:32.420596 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.420578 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d674s\"" Sep 09 20:51:32.420827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.420814 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 09 20:51:32.420879 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.420847 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 09 20:51:32.421070 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.421055 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 20:51:32.421115 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.421056 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 09 20:51:32.422639 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.422625 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 20:51:32.445793 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.445767 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9d949964-m9n92"] Sep 09 20:51:32.513653 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.513628 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg"] Sep 09 20:51:32.515278 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.515263 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.518130 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.518114 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 20:51:32.518245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.518131 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 20:51:32.518377 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.518364 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:51:32.518424 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.518387 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7qz8j\"" Sep 09 20:51:32.518584 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.518571 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 20:51:32.528772 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.528752 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg"] Sep 09 20:51:32.565445 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.565420 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-default-certificate\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.565535 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.565445 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6shqx\" (UniqueName: \"kubernetes.io/projected/e4728dbe-431a-44c6-93e1-889c4e86b95e-kube-api-access-6shqx\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.565535 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.565465 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.565535 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.565516 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.565648 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.565535 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-stats-auth\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.648141 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.648110 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:32.666766 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666718 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-default-certificate\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.666766 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666744 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6shqx\" (UniqueName: \"kubernetes.io/projected/e4728dbe-431a-44c6-93e1-889c4e86b95e-kube-api-access-6shqx\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.666894 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666764 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkh7b\" (UniqueName: \"kubernetes.io/projected/70979310-0c08-4686-b337-dc4dbca2b979-kube-api-access-fkh7b\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.666894 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666785 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.666894 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666809 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70979310-0c08-4686-b337-dc4dbca2b979-serving-cert\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.666894 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666848 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.666894 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666862 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-stats-auth\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.666894 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:32.666881 2538 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 20:51:32.667198 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:32.666943 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:33.166928014 +0000 UTC m=+234.785588224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : secret "router-metrics-certs-default" not found Sep 09 20:51:32.667198 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.666881 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70979310-0c08-4686-b337-dc4dbca2b979-config\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.667198 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:32.666992 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:33.166973315 +0000 UTC m=+234.785633526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : configmap references non-existent config key: service-ca.crt Sep 09 20:51:32.669269 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.669249 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-default-certificate\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.669370 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.669348 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-stats-auth\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.675440 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.675420 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6shqx\" (UniqueName: \"kubernetes.io/projected/e4728dbe-431a-44c6-93e1-889c4e86b95e-kube-api-access-6shqx\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:32.735294 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.735263 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:32.767282 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.767259 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkh7b\" (UniqueName: \"kubernetes.io/projected/70979310-0c08-4686-b337-dc4dbca2b979-kube-api-access-fkh7b\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.767353 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.767300 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70979310-0c08-4686-b337-dc4dbca2b979-serving-cert\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.767456 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.767437 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70979310-0c08-4686-b337-dc4dbca2b979-config\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.767899 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.767872 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70979310-0c08-4686-b337-dc4dbca2b979-config\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.769362 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.769343 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70979310-0c08-4686-b337-dc4dbca2b979-serving-cert\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.775144 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.775124 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkh7b\" (UniqueName: \"kubernetes.io/projected/70979310-0c08-4686-b337-dc4dbca2b979-kube-api-access-fkh7b\") pod \"service-ca-operator-859f94c4b-t2mvg\" (UID: \"70979310-0c08-4686-b337-dc4dbca2b979\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:32.823844 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:32.823822 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" Sep 09 20:51:33.089178 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:33.089108 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg"] Sep 09 20:51:33.092617 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:51:33.092592 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70979310_0c08_4686_b337_dc4dbca2b979.slice/crio-b9e53d0176287a3bda8cc90c899d66a3cbcb671b497e23084313260ab6417dbc WatchSource:0}: Error finding container b9e53d0176287a3bda8cc90c899d66a3cbcb671b497e23084313260ab6417dbc: Status 404 returned error can't find the container with id b9e53d0176287a3bda8cc90c899d66a3cbcb671b497e23084313260ab6417dbc Sep 09 20:51:33.170523 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:33.170495 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:33.170636 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:33.170558 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:33.170685 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:33.170646 2538 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 20:51:33.170685 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:33.170672 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:34.170650603 +0000 UTC m=+235.789310814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : configmap references non-existent config key: service-ca.crt Sep 09 20:51:33.170772 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:33.170691 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:34.170680454 +0000 UTC m=+235.789340686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : secret "router-metrics-certs-default" not found Sep 09 20:51:33.576541 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:33.576503 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" event={"ID":"70979310-0c08-4686-b337-dc4dbca2b979","Type":"ContainerStarted","Data":"b9e53d0176287a3bda8cc90c899d66a3cbcb671b497e23084313260ab6417dbc"} Sep 09 20:51:34.178078 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:34.178032 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:34.178265 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:34.178112 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:34.178265 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:34.178179 2538 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 20:51:34.178346 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:34.178262 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:36.178241898 +0000 UTC m=+237.796902122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : secret "router-metrics-certs-default" not found Sep 09 20:51:34.178346 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:34.178286 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:36.178279691 +0000 UTC m=+237.796939902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : configmap references non-existent config key: service-ca.crt Sep 09 20:51:35.584005 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:35.583966 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" event={"ID":"70979310-0c08-4686-b337-dc4dbca2b979","Type":"ContainerStarted","Data":"2307276c60c6fd7b833dba8b214bccbf6a00dbd74f2e3de6240473f7ca1de8ae"} Sep 09 20:51:35.601069 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:35.601025 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" podStartSLOduration=1.64235334 podStartE2EDuration="3.601006036s" totalImagesPullingTime="1.958652696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:35.600769548 +0000 UTC m=+237.219429780" watchObservedRunningTime="2026-09-09 20:51:35.601006036 +0000 UTC m=+237.219666263" Sep 09 20:51:35.889587 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:35.889513 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:35.990546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:35.990499 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:36.196907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:36.196834 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:36.196907 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:36.196887 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:36.197070 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:36.196967 2538 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 20:51:36.197070 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:36.197021 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:40.197007566 +0000 UTC m=+241.815667790 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : configmap references non-existent config key: service-ca.crt Sep 09 20:51:36.197070 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:36.197037 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:40.19703 +0000 UTC m=+241.815690209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : secret "router-metrics-certs-default" not found Sep 09 20:51:36.341986 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:36.341956 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:36.442235 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:36.442188 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:36.600333 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:36.600300 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:36.703310 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:36.703281 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:36.946357 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:36.946282 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:37.051157 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.051124 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:37.074498 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.074472 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xqx4m"] Sep 09 20:51:37.076614 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.076593 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.079616 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.079593 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 20:51:37.079734 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.079616 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 20:51:37.079734 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.079616 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 20:51:37.079734 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.079618 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 20:51:37.079905 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.079687 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x54bc\"" Sep 09 20:51:37.091539 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.091519 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xqx4m"] Sep 09 20:51:37.126980 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.126952 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:37.204189 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.204138 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/171176d5-bac7-41bf-921d-b5e44858e61f-crio-socket\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.204189 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.204164 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/171176d5-bac7-41bf-921d-b5e44858e61f-data-volume\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.204189 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.204182 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2rvd\" (UniqueName: \"kubernetes.io/projected/171176d5-bac7-41bf-921d-b5e44858e61f-kube-api-access-f2rvd\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.204357 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.204208 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.204357 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.204315 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.305408 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.305378 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.305549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.305429 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.305549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.305479 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/171176d5-bac7-41bf-921d-b5e44858e61f-crio-socket\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.305549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.305497 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/171176d5-bac7-41bf-921d-b5e44858e61f-data-volume\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.305549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.305511 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2rvd\" (UniqueName: \"kubernetes.io/projected/171176d5-bac7-41bf-921d-b5e44858e61f-kube-api-access-f2rvd\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.305549 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.305525 2538 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:51:37.305787 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.305570 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/171176d5-bac7-41bf-921d-b5e44858e61f-crio-socket\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.305787 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.305579 2538 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:51:37.305787 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.305587 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:37.805569166 +0000 UTC m=+239.424229389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : secret "insights-runtime-extractor-tls" not found Sep 09 20:51:37.305787 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.305635 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:37.805623969 +0000 UTC m=+239.424284179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : configmap "kube-rbac-proxy" not found Sep 09 20:51:37.305787 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.305747 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/171176d5-bac7-41bf-921d-b5e44858e61f-data-volume\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.314992 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.314971 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2rvd\" (UniqueName: \"kubernetes.io/projected/171176d5-bac7-41bf-921d-b5e44858e61f-kube-api-access-f2rvd\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.810508 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.810460 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.810880 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:37.810552 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:37.810880 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.810600 2538 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:51:37.810880 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.810652 2538 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:51:37.810880 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.810681 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:38.810663561 +0000 UTC m=+240.429323775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : configmap "kube-rbac-proxy" not found Sep 09 20:51:37.810880 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:37.810697 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:38.810689704 +0000 UTC m=+240.429349913 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : secret "insights-runtime-extractor-tls" not found Sep 09 20:51:38.251974 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:38.251894 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:38.357239 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:38.357197 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:38.451110 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:38.451084 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:38.819242 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:38.819199 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:38.819636 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:38.819265 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:38.819636 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:38.819329 2538 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:51:38.819636 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:38.819368 2538 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:51:38.819636 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:38.819374 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:40.819360905 +0000 UTC m=+242.438021116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : configmap "kube-rbac-proxy" not found Sep 09 20:51:38.819636 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:38.819488 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:40.81947233 +0000 UTC m=+242.438132539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : secret "insights-runtime-extractor-tls" not found Sep 09 20:51:39.086324 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:39.086217 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:39.249115 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:39.249091 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wjgh2_03b239fd-3642-4f5d-9380-8c7823fd82ed/dns-node-resolver/0.log" Sep 09 20:51:40.229014 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:40.228980 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:40.229480 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:40.229055 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:40.229480 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:40.229145 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:48.229127865 +0000 UTC m=+249.847788075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : configmap references non-existent config key: service-ca.crt Sep 09 20:51:40.229480 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:40.229173 2538 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 20:51:40.229480 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:40.229245 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:48.229206347 +0000 UTC m=+249.847866561 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : secret "router-metrics-certs-default" not found Sep 09 20:51:40.249131 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:40.249111 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vgpth_f0933970-91fd-4048-b522-eaba643670bb/node-ca/0.log" Sep 09 20:51:40.648680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:40.648636 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-49xnp_ef7c014b-1fab-4d97-8831-095a7712283b/serve-healthcheck-canary/0.log" Sep 09 20:51:40.833329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:40.833285 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:40.833495 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:40.833348 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:40.833495 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:40.833424 2538 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:51:40.833495 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:40.833427 2538 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:51:40.833589 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:40.833472 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:44.833459858 +0000 UTC m=+246.452120068 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : configmap "kube-rbac-proxy" not found Sep 09 20:51:40.833589 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:40.833523 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:44.833511471 +0000 UTC m=+246.452171681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : secret "insights-runtime-extractor-tls" not found Sep 09 20:51:41.619727 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:41.619697 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:42.879125 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:42.879098 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:43.141021 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:43.140933 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:43.337721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:43.337672 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:43.438345 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:43.438265 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:43.541991 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:43.541964 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:43.638442 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:43.638411 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:44.386361 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:44.386329 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:44.774430 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:44.774340 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:44.863065 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:44.863034 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:44.863210 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:44.863108 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:44.863210 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:44.863156 2538 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:51:44.863210 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:44.863197 2538 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:51:44.863338 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:44.863216 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:52.863200093 +0000 UTC m=+254.481860302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : configmap "kube-rbac-proxy" not found Sep 09 20:51:44.863338 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:44.863255 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls podName:171176d5-bac7-41bf-921d-b5e44858e61f nodeName:}" failed. No retries permitted until 2026-09-09 20:51:52.863242303 +0000 UTC m=+254.481902513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xqx4m" (UID: "171176d5-bac7-41bf-921d-b5e44858e61f") : secret "insights-runtime-extractor-tls" not found Sep 09 20:51:44.970488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:44.970458 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:45.371675 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:45.371630 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:45.795093 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:45.795009 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:46.182024 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:46.181991 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:46.369866 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:46.369823 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:46.672969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:46.672934 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:46.774159 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:46.774122 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:48.289206 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:48.289171 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:48.289600 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:48.289263 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:48.289600 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:51:48.289392 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle podName:e4728dbe-431a-44c6-93e1-889c4e86b95e nodeName:}" failed. No retries permitted until 2026-09-09 20:52:04.289375707 +0000 UTC m=+265.908035919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle") pod "router-default-9d949964-m9n92" (UID: "e4728dbe-431a-44c6-93e1-889c4e86b95e") : configmap references non-existent config key: service-ca.crt Sep 09 20:51:48.291812 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:48.291786 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4728dbe-431a-44c6-93e1-889c4e86b95e-metrics-certs\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:51:49.171874 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:49.171838 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:49.272415 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:49.272387 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:49.606967 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:49.606932 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:49.673877 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:49.673841 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:49.779299 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:49.779270 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:49.974431 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:49.974351 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:50.524271 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:50.524217 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:50.607549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:50.607519 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:51:50.609894 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:50.609871 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e-metrics-certs\") pod \"network-metrics-daemon-84ccl\" (UID: \"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e\") " pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:51:50.867895 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:50.867858 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-slrrp\"" Sep 09 20:51:50.875292 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:50.875273 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84ccl" Sep 09 20:51:51.163872 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:51.163807 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-84ccl"] Sep 09 20:51:51.166864 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:51:51.166827 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f32e5e_7e01_41aa_b6fd_fd2f272d3a4e.slice/crio-0a08c1504ac2f98754c5217dc570a3b8639e95611d9d09a2ccb1dabde9ae0bd6 WatchSource:0}: Error finding container 0a08c1504ac2f98754c5217dc570a3b8639e95611d9d09a2ccb1dabde9ae0bd6: Status 404 returned error can't find the container with id 0a08c1504ac2f98754c5217dc570a3b8639e95611d9d09a2ccb1dabde9ae0bd6 Sep 09 20:51:51.194195 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:51.194170 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:51.375180 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:51.375152 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:51.582214 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:51.582185 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:51.628580 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:51.628550 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84ccl" event={"ID":"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e","Type":"ContainerStarted","Data":"0a08c1504ac2f98754c5217dc570a3b8639e95611d9d09a2ccb1dabde9ae0bd6"} Sep 09 20:51:51.787770 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:51.787732 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:51.978523 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:51.978434 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:52.922072 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:52.922044 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:52.922525 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:52.922119 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:52.922688 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:52.922664 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/171176d5-bac7-41bf-921d-b5e44858e61f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:52.924672 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:52.924652 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/171176d5-bac7-41bf-921d-b5e44858e61f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xqx4m\" (UID: \"171176d5-bac7-41bf-921d-b5e44858e61f\") " pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:52.972889 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:52.972852 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:52.988763 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:52.988740 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x54bc\"" Sep 09 20:51:52.995997 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:52.995961 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xqx4m" Sep 09 20:51:53.171532 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.171458 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:53.273081 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.273016 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xqx4m"] Sep 09 20:51:53.276196 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:51:53.276154 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171176d5_bac7_41bf_921d_b5e44858e61f.slice/crio-ff348db31951e418232788260711e4c00512a7061054b14398c21e7cf8e40407 WatchSource:0}: Error finding container ff348db31951e418232788260711e4c00512a7061054b14398c21e7cf8e40407: Status 404 returned error can't find the container with id ff348db31951e418232788260711e4c00512a7061054b14398c21e7cf8e40407 Sep 09 20:51:53.401296 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.401263 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:53.575339 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.575305 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:53.639501 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.639466 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xqx4m" event={"ID":"171176d5-bac7-41bf-921d-b5e44858e61f","Type":"ContainerStarted","Data":"b400440953edb57d6b30360790949331a6069f4c0974bd0effc9dd147e855512"} Sep 09 20:51:53.639642 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.639548 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xqx4m" event={"ID":"171176d5-bac7-41bf-921d-b5e44858e61f","Type":"ContainerStarted","Data":"ff348db31951e418232788260711e4c00512a7061054b14398c21e7cf8e40407"} Sep 09 20:51:53.641167 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.641142 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84ccl" event={"ID":"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e","Type":"ContainerStarted","Data":"3fa6a630c333e384d857c0642583dae7cbddb0ddbf5e133eda0635ffc61df8aa"} Sep 09 20:51:53.641317 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.641184 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84ccl" event={"ID":"b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e","Type":"ContainerStarted","Data":"3703ea504327589826db9ba26dfc1ee94c0ceae86eb0e0efbe09ff9c545ebe9b"} Sep 09 20:51:53.659467 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:53.659426 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-84ccl" podStartSLOduration=252.54858055 podStartE2EDuration="4m14.659413784s" totalImagesPullingTime="2.110833234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:53.658453594 +0000 UTC m=+255.277113828" watchObservedRunningTime="2026-09-09 20:51:53.659413784 +0000 UTC m=+255.278074016" Sep 09 20:51:54.645920 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:54.645887 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xqx4m" event={"ID":"171176d5-bac7-41bf-921d-b5e44858e61f","Type":"ContainerStarted","Data":"32c493bef51b5dc78bae5e0b6ece40377eebf1dce8ca242b0f55b5cc2872b216"} Sep 09 20:51:56.652996 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:56.652961 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xqx4m" event={"ID":"171176d5-bac7-41bf-921d-b5e44858e61f","Type":"ContainerStarted","Data":"ae683146672daaa4525f06ea87ede399c4aa67e3cda3adad31ffd550c17cb4e4"} Sep 09 20:51:56.671844 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:56.671797 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xqx4m" podStartSLOduration=17.392469005 podStartE2EDuration="19.671782491s" totalImagesPullingTime="2.279313486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:51:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:56.669908851 +0000 UTC m=+258.288569087" watchObservedRunningTime="2026-09-09 20:51:56.671782491 +0000 UTC m=+258.290442722" Sep 09 20:51:58.526153 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.526119 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d8577cd6c-8wblz"] Sep 09 20:51:58.528143 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.528127 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.530855 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.530835 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 20:51:58.531019 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.531001 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nwrwc\"" Sep 09 20:51:58.531094 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.531049 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 20:51:58.531938 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.531917 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 20:51:58.536313 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.536293 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 20:51:58.549074 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.549054 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d8577cd6c-8wblz"] Sep 09 20:51:58.557727 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557698 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-bound-sa-token\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.557834 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557742 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac22653-b227-4818-99c2-a493ca64791c-registry-certificates\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.557834 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557773 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-registry-tls\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.557834 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557816 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44p8k\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-kube-api-access-44p8k\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.557999 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557844 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac22653-b227-4818-99c2-a493ca64791c-installation-pull-secrets\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.557999 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557893 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac22653-b227-4818-99c2-a493ca64791c-trusted-ca\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.557999 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557955 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac22653-b227-4818-99c2-a493ca64791c-ca-trust-extracted\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.557999 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.557988 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac22653-b227-4818-99c2-a493ca64791c-image-registry-private-configuration\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.568747 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.568719 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:58.658717 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658687 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac22653-b227-4818-99c2-a493ca64791c-registry-certificates\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.658887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658723 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-registry-tls\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.658887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658765 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44p8k\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-kube-api-access-44p8k\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.658887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658791 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac22653-b227-4818-99c2-a493ca64791c-installation-pull-secrets\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.658887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658831 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac22653-b227-4818-99c2-a493ca64791c-trusted-ca\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.658887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658862 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac22653-b227-4818-99c2-a493ca64791c-ca-trust-extracted\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.659134 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658893 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac22653-b227-4818-99c2-a493ca64791c-image-registry-private-configuration\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.659134 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.658934 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-bound-sa-token\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.659486 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.659454 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ac22653-b227-4818-99c2-a493ca64791c-ca-trust-extracted\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.659870 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.659811 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ac22653-b227-4818-99c2-a493ca64791c-registry-certificates\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.659981 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.659931 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ac22653-b227-4818-99c2-a493ca64791c-trusted-ca\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.661469 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.661442 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ac22653-b227-4818-99c2-a493ca64791c-installation-pull-secrets\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.661546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.661506 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-registry-tls\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.661546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.661513 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ac22653-b227-4818-99c2-a493ca64791c-image-registry-private-configuration\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.669162 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.669140 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44p8k\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-kube-api-access-44p8k\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.682931 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.682908 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:58.695136 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.695114 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ac22653-b227-4818-99c2-a493ca64791c-bound-sa-token\") pod \"image-registry-6d8577cd6c-8wblz\" (UID: \"9ac22653-b227-4818-99c2-a493ca64791c\") " pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:58.790040 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.789978 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:58.836984 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:58.836959 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:59.064347 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.064315 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.111048 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.111021 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d8577cd6c-8wblz"] Sep 09 20:51:59.114018 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:51:59.113994 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac22653_b227_4818_99c2_a493ca64791c.slice/crio-9be025ca979691b16a6396e38b6811f67b408b0b8489fcd8c23817eec73845f5 WatchSource:0}: Error finding container 9be025ca979691b16a6396e38b6811f67b408b0b8489fcd8c23817eec73845f5: Status 404 returned error can't find the container with id 9be025ca979691b16a6396e38b6811f67b408b0b8489fcd8c23817eec73845f5 Sep 09 20:51:59.165762 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.165735 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.265478 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.265444 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.539725 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.539646 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.603624 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.603595 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.663273 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.663246 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" event={"ID":"9ac22653-b227-4818-99c2-a493ca64791c","Type":"ContainerStarted","Data":"0a393cf77f5448b85aa704f7d7b23d4a3002bcece2f7d3078eb9e7938a77ab22"} Sep 09 20:51:59.663419 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.663298 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" event={"ID":"9ac22653-b227-4818-99c2-a493ca64791c","Type":"ContainerStarted","Data":"9be025ca979691b16a6396e38b6811f67b408b0b8489fcd8c23817eec73845f5"} Sep 09 20:51:59.663419 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.663382 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:51:59.687094 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.687050 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" podStartSLOduration=1.6870347589999999 podStartE2EDuration="1.687034759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:59.686374434 +0000 UTC m=+261.305034667" watchObservedRunningTime="2026-09-09 20:51:59.687034759 +0000 UTC m=+261.305695017" Sep 09 20:51:59.713882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.713846 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.777924 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.777892 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.909976 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.909911 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:51:59.991147 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:51:59.991119 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:00.728472 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:00.728444 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:00.999253 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:00.999163 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:01.100052 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:01.100018 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:01.201202 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:01.201170 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:01.543934 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:01.543905 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:01.803386 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:01.803359 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:03.400174 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:03.400139 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:03.708803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:03.708715 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:04.298576 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:04.298543 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:52:04.299116 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:04.299097 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4728dbe-431a-44c6-93e1-889c4e86b95e-service-ca-bundle\") pod \"router-default-9d949964-m9n92\" (UID: \"e4728dbe-431a-44c6-93e1-889c4e86b95e\") " pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:52:04.528841 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:04.528812 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d674s\"" Sep 09 20:52:04.536978 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:04.536953 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:52:04.809795 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:04.809764 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9d949964-m9n92"] Sep 09 20:52:04.812849 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:52:04.812818 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4728dbe_431a_44c6_93e1_889c4e86b95e.slice/crio-ea5d0db858a32383057a3b89f968e87ce99ed16713539c090d20a6ecb76ad0fc WatchSource:0}: Error finding container ea5d0db858a32383057a3b89f968e87ce99ed16713539c090d20a6ecb76ad0fc: Status 404 returned error can't find the container with id ea5d0db858a32383057a3b89f968e87ce99ed16713539c090d20a6ecb76ad0fc Sep 09 20:52:05.682479 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:05.682441 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9d949964-m9n92" event={"ID":"e4728dbe-431a-44c6-93e1-889c4e86b95e","Type":"ContainerStarted","Data":"63ed46e2d1eae027047656a51f8855eb53200ef6b785a821a0c374767ebe3ae7"} Sep 09 20:52:05.682844 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:05.682497 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9d949964-m9n92" event={"ID":"e4728dbe-431a-44c6-93e1-889c4e86b95e","Type":"ContainerStarted","Data":"ea5d0db858a32383057a3b89f968e87ce99ed16713539c090d20a6ecb76ad0fc"} Sep 09 20:52:05.702053 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:05.702006 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9d949964-m9n92" podStartSLOduration=33.701991985 podStartE2EDuration="33.701991985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:05.701412456 +0000 UTC m=+267.320072687" watchObservedRunningTime="2026-09-09 20:52:05.701991985 +0000 UTC m=+267.320652216" Sep 09 20:52:06.537587 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:06.537556 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:52:06.540132 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:06.540110 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:52:06.685344 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:06.685316 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:52:06.686539 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:06.686522 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9d949964-m9n92" Sep 09 20:52:06.965013 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:06.964971 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:07.063616 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:07.063583 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:07.131410 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:07.131358 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:07.357424 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:07.357382 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:07.554543 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:07.554494 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:07.654456 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:07.654380 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:09.957470 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:09.957438 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:10.023402 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:10.023373 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:16.540094 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:16.540057 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:16.646769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:16.646737 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:18.151525 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:18.151489 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:18.227444 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:18.227418 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:18.300957 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:18.300930 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:18.403196 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:18.403123 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:18.464582 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:18.464557 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:18.841288 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:18.841259 2538 patch_prober.go:28] interesting pod/image-registry-6d8577cd6c-8wblz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 20:52:18.841437 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:18.841310 2538 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" podUID="9ac22653-b227-4818-99c2-a493ca64791c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 20:52:19.370204 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:52:19.370160 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-m2r86" podUID="88c681c3-1804-4094-bb89-687d2fdb9b8a" Sep 09 20:52:19.725617 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:19.725525 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m2r86" Sep 09 20:52:20.671182 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:20.671154 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d8577cd6c-8wblz" Sep 09 20:52:20.832567 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:20.832527 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:20.930186 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:20.930106 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:21.091148 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:21.091110 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.130145 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.130113 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.204837 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.204809 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.305147 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.305108 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.516718 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.516635 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.599892 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.599861 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9csq9"] Sep 09 20:52:22.604418 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.604400 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.608324 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.608297 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 09 20:52:22.609790 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.609772 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 20:52:22.609790 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.609794 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 20:52:22.609964 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.609805 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 09 20:52:22.609964 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.609810 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 20:52:22.609964 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.609796 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 09 20:52:22.609964 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.609809 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8bjvn\"" Sep 09 20:52:22.619706 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.619681 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.621161 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.621143 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9csq9"] Sep 09 20:52:22.622007 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.621987 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fznkn"] Sep 09 20:52:22.625134 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.625116 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.628513 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.628496 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 20:52:22.628611 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.628566 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 20:52:22.628718 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.628703 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hgk6b\"" Sep 09 20:52:22.628953 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.628933 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 20:52:22.696314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.696283 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.723159 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723142 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxmgf\" (UniqueName: \"kubernetes.io/projected/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-kube-api-access-pxmgf\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723167 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-wtmp\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723187 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/887c8b15-bdcd-417c-bb15-d24a989799ea-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.723281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723242 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhfpw\" (UniqueName: \"kubernetes.io/projected/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-api-access-rhfpw\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.723281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723274 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-root\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723291 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-tls\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723310 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-sys\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723329 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/887c8b15-bdcd-417c-bb15-d24a989799ea-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.723432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723349 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.723432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723371 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.723432 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723425 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.723605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723453 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-accelerators-collector-config\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723487 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-metrics-client-ca\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723516 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:52:22.723605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723540 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-textfile\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.723605 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.723559 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.725788 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.725770 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88c681c3-1804-4094-bb89-687d2fdb9b8a-metrics-tls\") pod \"dns-default-m2r86\" (UID: \"88c681c3-1804-4094-bb89-687d2fdb9b8a\") " pod="openshift-dns/dns-default-m2r86" Sep 09 20:52:22.728683 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.728665 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5nprp\"" Sep 09 20:52:22.736908 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.736893 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m2r86" Sep 09 20:52:22.824908 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.824876 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhfpw\" (UniqueName: \"kubernetes.io/projected/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-api-access-rhfpw\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.824908 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.824912 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-root\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.824936 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-tls\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.824956 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-sys\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.824972 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/887c8b15-bdcd-417c-bb15-d24a989799ea-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.825076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.824995 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.825076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825015 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-root\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825024 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-sys\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825025 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.825281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825140 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.825281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825178 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-accelerators-collector-config\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825213 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-metrics-client-ca\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825281 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825269 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-textfile\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825296 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825463 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:52:22.825321 2538 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 09 20:52:22.825463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825346 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxmgf\" (UniqueName: \"kubernetes.io/projected/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-kube-api-access-pxmgf\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825463 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:52:22.825386 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-tls podName:887c8b15-bdcd-417c-bb15-d24a989799ea nodeName:}" failed. No retries permitted until 2026-09-09 20:52:23.325364768 +0000 UTC m=+284.944024978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-9csq9" (UID: "887c8b15-bdcd-417c-bb15-d24a989799ea") : secret "kube-state-metrics-tls" not found Sep 09 20:52:22.825463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825405 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-wtmp\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.825463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825439 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/887c8b15-bdcd-417c-bb15-d24a989799ea-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.825928 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825459 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/887c8b15-bdcd-417c-bb15-d24a989799ea-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.825928 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825800 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.825928 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825825 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-textfile\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.826083 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.825978 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-wtmp\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.826143 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.826089 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/887c8b15-bdcd-417c-bb15-d24a989799ea-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.826259 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.826212 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-metrics-client-ca\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.826482 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.826461 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-accelerators-collector-config\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.827619 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.827583 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.827619 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.827596 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-tls\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.827906 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.827883 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.836504 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.836475 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxmgf\" (UniqueName: \"kubernetes.io/projected/1aeaf8df-597d-405c-b0ff-ea832b3cdefe-kube-api-access-pxmgf\") pod \"node-exporter-fznkn\" (UID: \"1aeaf8df-597d-405c-b0ff-ea832b3cdefe\") " pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.836919 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.836901 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhfpw\" (UniqueName: \"kubernetes.io/projected/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-api-access-rhfpw\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:22.862074 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.862049 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:22.933486 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.933455 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fznkn" Sep 09 20:52:22.944647 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:52:22.944623 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aeaf8df_597d_405c_b0ff_ea832b3cdefe.slice/crio-f9cf4a5844ca3be6bfd2192b26c5da265f0a66aa04964b7318bf08e695dd6882 WatchSource:0}: Error finding container f9cf4a5844ca3be6bfd2192b26c5da265f0a66aa04964b7318bf08e695dd6882: Status 404 returned error can't find the container with id f9cf4a5844ca3be6bfd2192b26c5da265f0a66aa04964b7318bf08e695dd6882 Sep 09 20:52:22.964159 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:22.964131 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:23.011606 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.011546 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m2r86"] Sep 09 20:52:23.014836 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:52:23.014810 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c681c3_1804_4094_bb89_687d2fdb9b8a.slice/crio-cc664e542fb3eef968ccc05635a372f2410c776106870700de1689208a98cfef WatchSource:0}: Error finding container cc664e542fb3eef968ccc05635a372f2410c776106870700de1689208a98cfef: Status 404 returned error can't find the container with id cc664e542fb3eef968ccc05635a372f2410c776106870700de1689208a98cfef Sep 09 20:52:23.062194 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.062162 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:23.168315 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.168279 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:23.273329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.273294 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:23.329645 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.329582 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:23.331965 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.331939 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/887c8b15-bdcd-417c-bb15-d24a989799ea-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-9csq9\" (UID: \"887c8b15-bdcd-417c-bb15-d24a989799ea\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:23.513217 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.513194 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" Sep 09 20:52:23.605027 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.604961 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:23.703393 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.703368 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:23.739256 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.739207 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m2r86" event={"ID":"88c681c3-1804-4094-bb89-687d2fdb9b8a","Type":"ContainerStarted","Data":"cc664e542fb3eef968ccc05635a372f2410c776106870700de1689208a98cfef"} Sep 09 20:52:23.740251 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.740211 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fznkn" event={"ID":"1aeaf8df-597d-405c-b0ff-ea832b3cdefe","Type":"ContainerStarted","Data":"f9cf4a5844ca3be6bfd2192b26c5da265f0a66aa04964b7318bf08e695dd6882"} Sep 09 20:52:23.797851 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.797793 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-9csq9"] Sep 09 20:52:23.800484 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:52:23.800448 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod887c8b15_bdcd_417c_bb15_d24a989799ea.slice/crio-e079c1e5cadbc659ce176b0b48f0fbfeb2d569685fd8f6d8f10553f5d2a1d5ed WatchSource:0}: Error finding container e079c1e5cadbc659ce176b0b48f0fbfeb2d569685fd8f6d8f10553f5d2a1d5ed: Status 404 returned error can't find the container with id e079c1e5cadbc659ce176b0b48f0fbfeb2d569685fd8f6d8f10553f5d2a1d5ed Sep 09 20:52:23.852379 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:23.852333 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:24.001814 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:24.001694 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:24.745665 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:24.745630 2538 generic.go:356] "Generic (PLEG): container finished" podID="1aeaf8df-597d-405c-b0ff-ea832b3cdefe" containerID="202a1acfe98543c4117e8d397a3b0cf5b7ac70b73e8616d8c4e933e4c7eff600" exitCode=0 Sep 09 20:52:24.746141 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:24.745763 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fznkn" event={"ID":"1aeaf8df-597d-405c-b0ff-ea832b3cdefe","Type":"ContainerDied","Data":"202a1acfe98543c4117e8d397a3b0cf5b7ac70b73e8616d8c4e933e4c7eff600"} Sep 09 20:52:24.746994 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:24.746967 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" event={"ID":"887c8b15-bdcd-417c-bb15-d24a989799ea","Type":"ContainerStarted","Data":"e079c1e5cadbc659ce176b0b48f0fbfeb2d569685fd8f6d8f10553f5d2a1d5ed"} Sep 09 20:52:25.367245 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.367194 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:25.631922 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.631750 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:25.712808 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.712780 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:25.754273 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.754244 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fznkn" event={"ID":"1aeaf8df-597d-405c-b0ff-ea832b3cdefe","Type":"ContainerStarted","Data":"bd5e3ed47fcc1cc987fddae1ba3a1f39bff05035fdb1d30b0d05678550ebe7a1"} Sep 09 20:52:25.754594 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.754304 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fznkn" event={"ID":"1aeaf8df-597d-405c-b0ff-ea832b3cdefe","Type":"ContainerStarted","Data":"cffdeefbf985acdc1e54af74acef6d46812fc3eab1b217696851b4f0d4b0464f"} Sep 09 20:52:25.784747 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.784703 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fznkn" podStartSLOduration=2.389594157 podStartE2EDuration="3.78468788s" totalImagesPullingTime="1.314414637s" totalInitContainerRuntime="80.679086ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:25.782897813 +0000 UTC m=+287.401558046" watchObservedRunningTime="2026-09-09 20:52:25.78468788 +0000 UTC m=+287.403348160" Sep 09 20:52:25.795567 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.795421 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:25.891255 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.890616 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:25.987550 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:25.987507 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:26.759502 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.759469 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m2r86" event={"ID":"88c681c3-1804-4094-bb89-687d2fdb9b8a","Type":"ContainerStarted","Data":"636444ae02f6deba8f8bb8107af2820b53204fa49469cca138a54a1d9d202f63"} Sep 09 20:52:26.759882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.759526 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m2r86" event={"ID":"88c681c3-1804-4094-bb89-687d2fdb9b8a","Type":"ContainerStarted","Data":"31b886a6a0b71078d4f3bff1b2518af7f4e97d8e1262d61f31bfd3339071471b"} Sep 09 20:52:26.759882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.759554 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m2r86" Sep 09 20:52:26.761703 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.761678 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" event={"ID":"887c8b15-bdcd-417c-bb15-d24a989799ea","Type":"ContainerStarted","Data":"ca10367ed99575f1631842e1ae42504c160f2b45eb8aa8d85acd7319f5fac077"} Sep 09 20:52:26.761794 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.761724 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" event={"ID":"887c8b15-bdcd-417c-bb15-d24a989799ea","Type":"ContainerStarted","Data":"7d11cd3195b8f465a0e22c1c0df860660fc914a3b94005e2bd6b049937405ad2"} Sep 09 20:52:26.761794 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.761744 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" event={"ID":"887c8b15-bdcd-417c-bb15-d24a989799ea","Type":"ContainerStarted","Data":"04259e9358914df16178eb4f7bafc9d4a3caafcdc74ef1f3e1b9bd2432606bae"} Sep 09 20:52:26.776858 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.776813 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m2r86" podStartSLOduration=252.082310906 podStartE2EDuration="4m14.776799141s" totalImagesPullingTime="2.694488235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:26.776079033 +0000 UTC m=+288.394739290" watchObservedRunningTime="2026-09-09 20:52:26.776799141 +0000 UTC m=+288.395459389" Sep 09 20:52:26.794103 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:26.794061 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-9csq9" podStartSLOduration=2.891572302 podStartE2EDuration="4.79404872s" totalImagesPullingTime="1.902476418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:26.793345746 +0000 UTC m=+288.412005982" watchObservedRunningTime="2026-09-09 20:52:26.79404872 +0000 UTC m=+288.412708951" Sep 09 20:52:27.005061 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.005027 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6bf66bb88d-n2nf7"] Sep 09 20:52:27.007994 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.007977 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.010908 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.010860 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 09 20:52:27.011806 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.011788 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 09 20:52:27.011894 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.011791 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-34qatu80ss827\"" Sep 09 20:52:27.011961 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.011951 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-k67r2\"" Sep 09 20:52:27.012189 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.012170 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 20:52:27.012287 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.012274 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 09 20:52:27.019116 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.019098 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6bf66bb88d-n2nf7"] Sep 09 20:52:27.060633 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.060593 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-client-ca-bundle\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.060633 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.060632 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f21313c6-3913-4f62-9e0c-dbac19ffbb94-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.060827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.060659 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f21313c6-3913-4f62-9e0c-dbac19ffbb94-audit-log\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.060827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.060726 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp9wk\" (UniqueName: \"kubernetes.io/projected/f21313c6-3913-4f62-9e0c-dbac19ffbb94-kube-api-access-cp9wk\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.060827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.060760 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-secret-metrics-server-tls\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.060827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.060785 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f21313c6-3913-4f62-9e0c-dbac19ffbb94-metrics-server-audit-profiles\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.060827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.060805 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-secret-metrics-server-client-certs\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.161520 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.161486 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp9wk\" (UniqueName: \"kubernetes.io/projected/f21313c6-3913-4f62-9e0c-dbac19ffbb94-kube-api-access-cp9wk\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.161707 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.161537 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-secret-metrics-server-tls\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.161707 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.161565 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f21313c6-3913-4f62-9e0c-dbac19ffbb94-metrics-server-audit-profiles\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.161830 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.161738 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-secret-metrics-server-client-certs\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.161830 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.161788 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-client-ca-bundle\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.161830 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.161822 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f21313c6-3913-4f62-9e0c-dbac19ffbb94-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.161985 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.161850 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f21313c6-3913-4f62-9e0c-dbac19ffbb94-audit-log\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.162243 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.162191 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f21313c6-3913-4f62-9e0c-dbac19ffbb94-audit-log\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.162600 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.162563 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f21313c6-3913-4f62-9e0c-dbac19ffbb94-metrics-server-audit-profiles\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.162688 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.162650 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f21313c6-3913-4f62-9e0c-dbac19ffbb94-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.164311 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.164290 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-secret-metrics-server-client-certs\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.164413 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.164370 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-client-ca-bundle\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.164518 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.164435 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f21313c6-3913-4f62-9e0c-dbac19ffbb94-secret-metrics-server-tls\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.170749 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.170726 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp9wk\" (UniqueName: \"kubernetes.io/projected/f21313c6-3913-4f62-9e0c-dbac19ffbb94-kube-api-access-cp9wk\") pod \"metrics-server-6bf66bb88d-n2nf7\" (UID: \"f21313c6-3913-4f62-9e0c-dbac19ffbb94\") " pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.317115 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.317084 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:27.353704 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.353671 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:27.422648 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.422616 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:27.495660 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.495627 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:27.597475 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.597410 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6bf66bb88d-n2nf7"] Sep 09 20:52:27.600414 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:52:27.600384 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21313c6_3913_4f62_9e0c_dbac19ffbb94.slice/crio-40c4232d311f2cdc312b80b5612d57d1fdf7783fe7077dc15110d8d8f1845de2 WatchSource:0}: Error finding container 40c4232d311f2cdc312b80b5612d57d1fdf7783fe7077dc15110d8d8f1845de2: Status 404 returned error can't find the container with id 40c4232d311f2cdc312b80b5612d57d1fdf7783fe7077dc15110d8d8f1845de2 Sep 09 20:52:27.766593 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:27.766553 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" event={"ID":"f21313c6-3913-4f62-9e0c-dbac19ffbb94","Type":"ContainerStarted","Data":"40c4232d311f2cdc312b80b5612d57d1fdf7783fe7077dc15110d8d8f1845de2"} Sep 09 20:52:28.347749 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.347715 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:28.486942 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.486904 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:28.811481 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.811445 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:28.815720 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.815696 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:52:28.819252 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.819213 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.822102 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.822080 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 20:52:28.822341 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.822323 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 20:52:28.822413 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.822323 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 20:52:28.822620 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.822606 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 20:52:28.822923 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.822906 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p2pqg\"" Sep 09 20:52:28.823248 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.823212 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 20:52:28.823377 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.823361 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4b05mlmf8qke\"" Sep 09 20:52:28.823916 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.823897 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 20:52:28.824004 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.823979 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 20:52:28.824047 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.823897 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 20:52:28.824125 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.824104 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 20:52:28.824492 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.824324 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 20:52:28.824492 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.824357 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 20:52:28.824492 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.824370 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 20:52:28.824492 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.824237 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 20:52:28.826670 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.826645 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 20:52:28.841166 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.841142 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:52:28.875743 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.875710 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-web-config\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.875898 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.875758 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.875898 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.875815 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.875898 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.875848 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876070 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.875917 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876070 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.875961 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876070 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.875986 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q65fl\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-kube-api-access-q65fl\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876070 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876042 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config-out\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876113 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876139 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876161 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876189 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876215 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876272 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876314 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876300 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876644 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876324 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876644 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876445 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876644 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876472 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876644 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876530 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.876644 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.876573 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.977835 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.977799 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978013 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.977853 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978096 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978040 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978150 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978117 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978203 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978157 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978203 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978172 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q65fl\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-kube-api-access-q65fl\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978340 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978203 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config-out\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978340 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978246 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978340 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978269 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978340 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978286 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978340 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978302 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978340 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978324 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978348 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978370 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978391 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978428 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978445 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978462 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978494 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978623 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978536 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-web-config\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.978995 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.978970 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.979174 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.979146 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.979302 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.979280 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.980583 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.980551 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.982257 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.982037 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.982257 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.982067 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.983375 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.982748 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.983375 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.982771 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.983375 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.983024 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.983375 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.983328 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.985033 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.984153 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config-out\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.985033 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.984562 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.985033 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.984782 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.985033 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.984933 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-web-config\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.985033 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.984991 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.985680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.985656 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.986027 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.986006 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.986100 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.986045 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.987201 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.987177 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:28.987384 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:28.987365 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q65fl\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-kube-api-access-q65fl\") pod \"prometheus-k8s-0\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:29.131619 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:29.131532 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:29.494523 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:29.494445 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:52:29.497293 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:52:29.497270 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e76bf5e_7bcf_4221_a20a_9237d557bb88.slice/crio-77e4680a6c958ce2fa9d0544c41d8d3abfa9ed594d502891a5775a1b9b99ca87 WatchSource:0}: Error finding container 77e4680a6c958ce2fa9d0544c41d8d3abfa9ed594d502891a5775a1b9b99ca87: Status 404 returned error can't find the container with id 77e4680a6c958ce2fa9d0544c41d8d3abfa9ed594d502891a5775a1b9b99ca87 Sep 09 20:52:29.777601 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:29.777500 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" event={"ID":"f21313c6-3913-4f62-9e0c-dbac19ffbb94","Type":"ContainerStarted","Data":"d58ba4f268c9549f7627478aa6ca1c32ebc8ad9e45091fca9657b19ddc1c9a87"} Sep 09 20:52:29.778783 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:29.778755 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerStarted","Data":"77e4680a6c958ce2fa9d0544c41d8d3abfa9ed594d502891a5775a1b9b99ca87"} Sep 09 20:52:29.795278 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:29.795204 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" podStartSLOduration=2.133882531 podStartE2EDuration="3.795185394s" totalImagesPullingTime="1.661302863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:29.79468723 +0000 UTC m=+291.413347486" watchObservedRunningTime="2026-09-09 20:52:29.795185394 +0000 UTC m=+291.413845626" Sep 09 20:52:30.784002 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:30.783964 2538 generic.go:356] "Generic (PLEG): container finished" podID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" exitCode=0 Sep 09 20:52:30.784393 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:30.784000 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289"} Sep 09 20:52:34.033787 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:34.033722 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:34.135089 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:34.134988 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:34.800838 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:34.800798 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerStarted","Data":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} Sep 09 20:52:34.801090 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:34.800874 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerStarted","Data":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} Sep 09 20:52:36.123368 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.123342 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:36.224962 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.224863 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:36.768924 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.768897 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m2r86" Sep 09 20:52:36.813242 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.813189 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerStarted","Data":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} Sep 09 20:52:36.813390 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.813299 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerStarted","Data":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} Sep 09 20:52:36.813390 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.813342 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerStarted","Data":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} Sep 09 20:52:36.813390 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.813381 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerStarted","Data":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} Sep 09 20:52:36.845133 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:36.845080 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.771895594 podStartE2EDuration="8.845063209s" totalImagesPullingTime="6.054151153s" totalInitContainerRuntime="19.016462ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:36.842687006 +0000 UTC m=+298.461347237" watchObservedRunningTime="2026-09-09 20:52:36.845063209 +0000 UTC m=+298.463723441" Sep 09 20:52:38.786326 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:38.786300 2538 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 20:52:39.132460 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:39.132430 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:52:43.050191 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:43.050156 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:43.182004 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:43.181960 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:43.276960 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:43.276926 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:43.378553 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:43.378477 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:43.526367 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:43.526339 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:45.967806 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:45.967771 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:46.068558 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:46.068529 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:47.317487 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:47.317449 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:47.317885 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:47.317519 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:52:48.398822 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:48.398787 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:52:48.471038 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:52:48.471011 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:03.445407 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:03.445363 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:03.539412 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:03.539381 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:05.920453 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:05.920376 2538 generic.go:356] "Generic (PLEG): container finished" podID="70979310-0c08-4686-b337-dc4dbca2b979" containerID="2307276c60c6fd7b833dba8b214bccbf6a00dbd74f2e3de6240473f7ca1de8ae" exitCode=0 Sep 09 20:53:05.920825 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:05.920453 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" event={"ID":"70979310-0c08-4686-b337-dc4dbca2b979","Type":"ContainerDied","Data":"2307276c60c6fd7b833dba8b214bccbf6a00dbd74f2e3de6240473f7ca1de8ae"} Sep 09 20:53:05.920825 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:05.920762 2538 scope.go:117] "RemoveContainer" containerID="2307276c60c6fd7b833dba8b214bccbf6a00dbd74f2e3de6240473f7ca1de8ae" Sep 09 20:53:06.925807 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:06.925777 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-t2mvg" event={"ID":"70979310-0c08-4686-b337-dc4dbca2b979","Type":"ContainerStarted","Data":"1a8dc9f26b5df3747b2bcd05f7d02bde063308fbf7c3d7190feda58c727c888c"} Sep 09 20:53:07.322828 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:07.322801 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:53:07.326697 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:07.326676 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6bf66bb88d-n2nf7" Sep 09 20:53:29.131906 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:29.131875 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:29.148937 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:29.148911 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:30.017138 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:30.017063 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:31.495591 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:31.495553 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:31.595324 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:31.595286 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:34.271991 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:34.271952 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:34.381712 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:34.381679 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:38.746803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:38.746760 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:38.845655 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:38.845626 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:47.405820 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.405790 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:53:47.406272 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.406080 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="prometheus" containerID="cri-o://c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" gracePeriod=600 Sep 09 20:53:47.406272 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.406119 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy" containerID="cri-o://d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" gracePeriod=600 Sep 09 20:53:47.406272 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.406130 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="thanos-sidecar" containerID="cri-o://0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" gracePeriod=600 Sep 09 20:53:47.406272 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.406152 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-web" containerID="cri-o://b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" gracePeriod=600 Sep 09 20:53:47.406272 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.406194 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" gracePeriod=600 Sep 09 20:53:47.406504 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.406197 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="config-reloader" containerID="cri-o://edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" gracePeriod=600 Sep 09 20:53:47.651151 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.651129 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:47.705950 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.705879 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.705950 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.705922 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-thanos-prometheus-http-client-file\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.705950 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.705947 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-db\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.706165 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.705964 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-grpc-tls\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.706165 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.705985 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-tls\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.706165 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706005 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-kubelet-serving-ca-bundle\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.706165 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706031 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-web-config\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.706165 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706064 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-metrics-client-certs\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.706165 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706091 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-tls-assets\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706673 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-2\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706711 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config-out\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706735 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706764 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q65fl\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-kube-api-access-q65fl\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706788 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-metrics-client-ca\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706819 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-0\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706842 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-serving-certs-ca-bundle\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706872 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-trusted-ca-bundle\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706896 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-kube-rbac-proxy\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706937 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-1\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.706992 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\" (UID: \"4e76bf5e-7bcf-4221-a20a-9237d557bb88\") " Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.707538 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:53:47.708304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.708044 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:53:47.709400 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.708675 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:53:47.709400 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.708782 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:53:47.709400 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.708829 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:53:47.709400 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.709202 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:53:47.709400 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.709290 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.709666 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.709435 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.709666 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.709572 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.710010 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.709953 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.710358 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.710328 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:53:47.710998 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.710975 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:53:47.711560 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.711343 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.711642 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.711618 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.712730 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.712696 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.712830 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.712743 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config" (OuterVolumeSpecName: "config") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.712898 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.712847 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:53:47.713492 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.713469 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-kube-api-access-q65fl" (OuterVolumeSpecName: "kube-api-access-q65fl") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "kube-api-access-q65fl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:53:47.713718 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.713703 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config-out" (OuterVolumeSpecName: "config-out") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:53:47.723325 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.723301 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-web-config" (OuterVolumeSpecName: "web-config") pod "4e76bf5e-7bcf-4221-a20a-9237d557bb88" (UID: "4e76bf5e-7bcf-4221-a20a-9237d557bb88"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:53:47.808082 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808038 2538 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808082 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808077 2538 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config-out\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808082 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808085 2538 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808082 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808093 2538 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q65fl\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-kube-api-access-q65fl\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808101 2538 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-metrics-client-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808110 2538 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808119 2538 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808127 2538 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808135 2538 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-kube-rbac-proxy\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808143 2538 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808152 2538 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808160 2538 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808169 2538 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808176 2538 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4e76bf5e-7bcf-4221-a20a-9237d557bb88-prometheus-k8s-db\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808183 2538 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-grpc-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808191 2538 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808199 2538 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e76bf5e-7bcf-4221-a20a-9237d557bb88-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808253 2538 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-web-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808267 2538 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4e76bf5e-7bcf-4221-a20a-9237d557bb88-secret-metrics-client-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:47.808354 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:47.808279 2538 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e76bf5e-7bcf-4221-a20a-9237d557bb88-tls-assets\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:53:48.068076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068040 2538 generic.go:356] "Generic (PLEG): container finished" podID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" exitCode=0 Sep 09 20:53:48.068076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068069 2538 generic.go:356] "Generic (PLEG): container finished" podID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" exitCode=0 Sep 09 20:53:48.068076 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068079 2538 generic.go:356] "Generic (PLEG): container finished" podID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" exitCode=0 Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068091 2538 generic.go:356] "Generic (PLEG): container finished" podID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" exitCode=0 Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068099 2538 generic.go:356] "Generic (PLEG): container finished" podID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" exitCode=0 Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068108 2538 generic.go:356] "Generic (PLEG): container finished" podID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" exitCode=0 Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068128 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068120 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068305 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068354 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} Sep 09 20:53:48.068387 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068371 2538 scope.go:117] "RemoveContainer" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.068741 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068394 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} Sep 09 20:53:48.068741 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068437 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} Sep 09 20:53:48.068741 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068481 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} Sep 09 20:53:48.068741 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.068521 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4e76bf5e-7bcf-4221-a20a-9237d557bb88","Type":"ContainerDied","Data":"77e4680a6c958ce2fa9d0544c41d8d3abfa9ed594d502891a5775a1b9b99ca87"} Sep 09 20:53:48.077472 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.077448 2538 scope.go:117] "RemoveContainer" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.084724 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.084708 2538 scope.go:117] "RemoveContainer" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.091347 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.091325 2538 scope.go:117] "RemoveContainer" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.099216 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.099125 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:53:48.099347 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.099293 2538 scope.go:117] "RemoveContainer" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.105107 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.105083 2538 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:53:48.108042 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.108028 2538 scope.go:117] "RemoveContainer" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.117178 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.117161 2538 scope.go:117] "RemoveContainer" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.124260 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.124244 2538 scope.go:117] "RemoveContainer" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.124529 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:53:48.124502 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": container with ID starting with 0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91 not found: ID does not exist" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.124616 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.124538 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} err="failed to get container status \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": rpc error: code = NotFound desc = could not find container \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": container with ID starting with 0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91 not found: ID does not exist" Sep 09 20:53:48.124616 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.124573 2538 scope.go:117] "RemoveContainer" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.124836 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:53:48.124819 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": container with ID starting with d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77 not found: ID does not exist" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.124879 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.124841 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} err="failed to get container status \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": rpc error: code = NotFound desc = could not find container \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": container with ID starting with d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77 not found: ID does not exist" Sep 09 20:53:48.124879 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.124858 2538 scope.go:117] "RemoveContainer" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.125110 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:53:48.125094 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": container with ID starting with b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db not found: ID does not exist" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.125150 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125114 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} err="failed to get container status \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": rpc error: code = NotFound desc = could not find container \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": container with ID starting with b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db not found: ID does not exist" Sep 09 20:53:48.125150 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125127 2538 scope.go:117] "RemoveContainer" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.125374 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:53:48.125361 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": container with ID starting with 0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d not found: ID does not exist" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.125411 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125377 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} err="failed to get container status \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": rpc error: code = NotFound desc = could not find container \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": container with ID starting with 0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d not found: ID does not exist" Sep 09 20:53:48.125411 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125391 2538 scope.go:117] "RemoveContainer" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.125629 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:53:48.125614 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": container with ID starting with edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe not found: ID does not exist" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.125671 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125634 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} err="failed to get container status \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": rpc error: code = NotFound desc = could not find container \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": container with ID starting with edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe not found: ID does not exist" Sep 09 20:53:48.125671 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125647 2538 scope.go:117] "RemoveContainer" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.125895 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:53:48.125870 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": container with ID starting with c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570 not found: ID does not exist" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.125935 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125899 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} err="failed to get container status \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": rpc error: code = NotFound desc = could not find container \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": container with ID starting with c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570 not found: ID does not exist" Sep 09 20:53:48.125935 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.125913 2538 scope.go:117] "RemoveContainer" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.126159 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:53:48.126146 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": container with ID starting with 6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289 not found: ID does not exist" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.126201 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.126161 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289"} err="failed to get container status \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": rpc error: code = NotFound desc = could not find container \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": container with ID starting with 6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289 not found: ID does not exist" Sep 09 20:53:48.126201 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.126176 2538 scope.go:117] "RemoveContainer" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.126603 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.126578 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} err="failed to get container status \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": rpc error: code = NotFound desc = could not find container \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": container with ID starting with 0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91 not found: ID does not exist" Sep 09 20:53:48.126603 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.126603 2538 scope.go:117] "RemoveContainer" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.128728 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.128705 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} err="failed to get container status \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": rpc error: code = NotFound desc = could not find container \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": container with ID starting with d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77 not found: ID does not exist" Sep 09 20:53:48.128823 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.128729 2538 scope.go:117] "RemoveContainer" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.129150 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129130 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} err="failed to get container status \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": rpc error: code = NotFound desc = could not find container \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": container with ID starting with b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db not found: ID does not exist" Sep 09 20:53:48.129195 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129151 2538 scope.go:117] "RemoveContainer" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.129452 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129435 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} err="failed to get container status \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": rpc error: code = NotFound desc = could not find container \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": container with ID starting with 0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d not found: ID does not exist" Sep 09 20:53:48.129503 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129452 2538 scope.go:117] "RemoveContainer" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.129707 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129688 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} err="failed to get container status \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": rpc error: code = NotFound desc = could not find container \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": container with ID starting with edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe not found: ID does not exist" Sep 09 20:53:48.129756 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129708 2538 scope.go:117] "RemoveContainer" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.129975 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129954 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} err="failed to get container status \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": rpc error: code = NotFound desc = could not find container \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": container with ID starting with c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570 not found: ID does not exist" Sep 09 20:53:48.129975 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.129975 2538 scope.go:117] "RemoveContainer" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.130207 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130190 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289"} err="failed to get container status \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": rpc error: code = NotFound desc = could not find container \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": container with ID starting with 6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289 not found: ID does not exist" Sep 09 20:53:48.130267 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130207 2538 scope.go:117] "RemoveContainer" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.130457 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130440 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} err="failed to get container status \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": rpc error: code = NotFound desc = could not find container \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": container with ID starting with 0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91 not found: ID does not exist" Sep 09 20:53:48.130566 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130459 2538 scope.go:117] "RemoveContainer" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.130684 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130668 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} err="failed to get container status \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": rpc error: code = NotFound desc = could not find container \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": container with ID starting with d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77 not found: ID does not exist" Sep 09 20:53:48.130730 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130685 2538 scope.go:117] "RemoveContainer" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.130867 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130850 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} err="failed to get container status \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": rpc error: code = NotFound desc = could not find container \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": container with ID starting with b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db not found: ID does not exist" Sep 09 20:53:48.130908 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.130867 2538 scope.go:117] "RemoveContainer" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.131036 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131020 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} err="failed to get container status \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": rpc error: code = NotFound desc = could not find container \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": container with ID starting with 0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d not found: ID does not exist" Sep 09 20:53:48.131082 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131036 2538 scope.go:117] "RemoveContainer" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.131211 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131196 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} err="failed to get container status \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": rpc error: code = NotFound desc = could not find container \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": container with ID starting with edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe not found: ID does not exist" Sep 09 20:53:48.131277 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131211 2538 scope.go:117] "RemoveContainer" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.131407 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131393 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} err="failed to get container status \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": rpc error: code = NotFound desc = could not find container \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": container with ID starting with c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570 not found: ID does not exist" Sep 09 20:53:48.131449 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131408 2538 scope.go:117] "RemoveContainer" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.131628 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131612 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289"} err="failed to get container status \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": rpc error: code = NotFound desc = could not find container \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": container with ID starting with 6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289 not found: ID does not exist" Sep 09 20:53:48.131628 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131627 2538 scope.go:117] "RemoveContainer" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.131825 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131808 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} err="failed to get container status \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": rpc error: code = NotFound desc = could not find container \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": container with ID starting with 0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91 not found: ID does not exist" Sep 09 20:53:48.131865 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.131826 2538 scope.go:117] "RemoveContainer" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.132118 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132101 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} err="failed to get container status \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": rpc error: code = NotFound desc = could not find container \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": container with ID starting with d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77 not found: ID does not exist" Sep 09 20:53:48.132176 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132119 2538 scope.go:117] "RemoveContainer" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.132383 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132365 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} err="failed to get container status \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": rpc error: code = NotFound desc = could not find container \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": container with ID starting with b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db not found: ID does not exist" Sep 09 20:53:48.132435 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132385 2538 scope.go:117] "RemoveContainer" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.132586 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132571 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} err="failed to get container status \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": rpc error: code = NotFound desc = could not find container \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": container with ID starting with 0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d not found: ID does not exist" Sep 09 20:53:48.132625 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132586 2538 scope.go:117] "RemoveContainer" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.132830 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132810 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} err="failed to get container status \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": rpc error: code = NotFound desc = could not find container \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": container with ID starting with edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe not found: ID does not exist" Sep 09 20:53:48.132917 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.132832 2538 scope.go:117] "RemoveContainer" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.133110 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133086 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} err="failed to get container status \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": rpc error: code = NotFound desc = could not find container \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": container with ID starting with c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570 not found: ID does not exist" Sep 09 20:53:48.133110 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133110 2538 scope.go:117] "RemoveContainer" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.133451 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133426 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289"} err="failed to get container status \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": rpc error: code = NotFound desc = could not find container \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": container with ID starting with 6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289 not found: ID does not exist" Sep 09 20:53:48.133451 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133444 2538 scope.go:117] "RemoveContainer" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.133677 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133657 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} err="failed to get container status \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": rpc error: code = NotFound desc = could not find container \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": container with ID starting with 0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91 not found: ID does not exist" Sep 09 20:53:48.133727 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133679 2538 scope.go:117] "RemoveContainer" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.133906 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133887 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} err="failed to get container status \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": rpc error: code = NotFound desc = could not find container \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": container with ID starting with d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77 not found: ID does not exist" Sep 09 20:53:48.133954 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.133908 2538 scope.go:117] "RemoveContainer" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.134119 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134102 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} err="failed to get container status \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": rpc error: code = NotFound desc = could not find container \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": container with ID starting with b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db not found: ID does not exist" Sep 09 20:53:48.134159 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134119 2538 scope.go:117] "RemoveContainer" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.134403 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134384 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} err="failed to get container status \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": rpc error: code = NotFound desc = could not find container \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": container with ID starting with 0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d not found: ID does not exist" Sep 09 20:53:48.134403 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134403 2538 scope.go:117] "RemoveContainer" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.134626 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134611 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} err="failed to get container status \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": rpc error: code = NotFound desc = could not find container \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": container with ID starting with edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe not found: ID does not exist" Sep 09 20:53:48.134670 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134626 2538 scope.go:117] "RemoveContainer" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.134821 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134806 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} err="failed to get container status \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": rpc error: code = NotFound desc = could not find container \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": container with ID starting with c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570 not found: ID does not exist" Sep 09 20:53:48.134864 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.134821 2538 scope.go:117] "RemoveContainer" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.135034 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135016 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289"} err="failed to get container status \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": rpc error: code = NotFound desc = could not find container \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": container with ID starting with 6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289 not found: ID does not exist" Sep 09 20:53:48.135088 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135035 2538 scope.go:117] "RemoveContainer" containerID="0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91" Sep 09 20:53:48.135278 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135259 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91"} err="failed to get container status \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": rpc error: code = NotFound desc = could not find container \"0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91\": container with ID starting with 0a8c9f940239731d722e3edc8bd1e2adaf31c4eaa4ab29b22bd7d0ab84447b91 not found: ID does not exist" Sep 09 20:53:48.135278 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135278 2538 scope.go:117] "RemoveContainer" containerID="d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77" Sep 09 20:53:48.135549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135533 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77"} err="failed to get container status \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": rpc error: code = NotFound desc = could not find container \"d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77\": container with ID starting with d8781c902f8dc2db47de944df4d449efd154c2eee566d30555e1ea6ff299cb77 not found: ID does not exist" Sep 09 20:53:48.135549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135548 2538 scope.go:117] "RemoveContainer" containerID="b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db" Sep 09 20:53:48.135762 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135743 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db"} err="failed to get container status \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": rpc error: code = NotFound desc = could not find container \"b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db\": container with ID starting with b5e3aeb04b512f96a07312e531c63aa030e0ad7a2dff21c12662c1967009c7db not found: ID does not exist" Sep 09 20:53:48.135807 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135763 2538 scope.go:117] "RemoveContainer" containerID="0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d" Sep 09 20:53:48.135966 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135947 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d"} err="failed to get container status \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": rpc error: code = NotFound desc = could not find container \"0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d\": container with ID starting with 0131d66abf8de873ebf9187d30ab76c4f28318254df165be0dc47f7dd103b76d not found: ID does not exist" Sep 09 20:53:48.136004 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.135966 2538 scope.go:117] "RemoveContainer" containerID="edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe" Sep 09 20:53:48.136164 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.136140 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe"} err="failed to get container status \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": rpc error: code = NotFound desc = could not find container \"edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe\": container with ID starting with edc47c84e93cbc574fe2fefe16014e6faabc9d96c196b3fe10c2b032aa2db6fe not found: ID does not exist" Sep 09 20:53:48.136208 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.136164 2538 scope.go:117] "RemoveContainer" containerID="c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570" Sep 09 20:53:48.136404 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.136387 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570"} err="failed to get container status \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": rpc error: code = NotFound desc = could not find container \"c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570\": container with ID starting with c05c33fb7b1e21daefc1472f594acb5ca613373a61899faf8067b30a503b3570 not found: ID does not exist" Sep 09 20:53:48.136404 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.136404 2538 scope.go:117] "RemoveContainer" containerID="6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289" Sep 09 20:53:48.136607 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.136592 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289"} err="failed to get container status \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": rpc error: code = NotFound desc = could not find container \"6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289\": container with ID starting with 6443849c88cd0b6afd6a3209f74d219c360631f89ee8b96399a1bb2dbe2a7289 not found: ID does not exist" Sep 09 20:53:48.150286 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150268 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:53:48.150504 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150493 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="init-config-reloader" Sep 09 20:53:48.150546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150507 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="init-config-reloader" Sep 09 20:53:48.150546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150521 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="thanos-sidecar" Sep 09 20:53:48.150546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150526 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="thanos-sidecar" Sep 09 20:53:48.150546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150533 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-thanos" Sep 09 20:53:48.150546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150539 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-thanos" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150549 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150554 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150561 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-web" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150566 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-web" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150575 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="prometheus" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150580 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="prometheus" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150587 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="config-reloader" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150591 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="config-reloader" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150627 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="prometheus" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150634 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="thanos-sidecar" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150640 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-thanos" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150647 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150652 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="config-reloader" Sep 09 20:53:48.150680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.150657 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" containerName="kube-rbac-proxy-web" Sep 09 20:53:48.156175 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.156156 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.161031 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161010 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 20:53:48.161108 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161057 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 20:53:48.161108 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161019 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 20:53:48.161212 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161019 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 20:53:48.161295 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161279 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 20:53:48.161357 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161324 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 20:53:48.161667 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161651 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 20:53:48.161753 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.161737 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4b05mlmf8qke\"" Sep 09 20:53:48.162438 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.162419 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 20:53:48.162604 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.162587 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 20:53:48.162695 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.162663 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 20:53:48.164479 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.164461 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p2pqg\"" Sep 09 20:53:48.164580 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.164568 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 20:53:48.169012 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.168998 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 20:53:48.170932 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.170917 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 20:53:48.173424 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.173410 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 20:53:48.212327 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212302 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212408 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212336 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212408 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212353 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212408 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212371 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212405 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212436 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212454 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212486 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212504 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212518 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212533 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212599 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212618 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212633 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5nqc\" (UniqueName: \"kubernetes.io/projected/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-kube-api-access-b5nqc\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212652 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-config\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212665 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212682 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212698 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212954 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212719 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.212954 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.212747 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.222035 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.222011 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:53:48.314097 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314067 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314290 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314105 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314290 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314125 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314290 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314259 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314312 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314336 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314356 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5nqc\" (UniqueName: \"kubernetes.io/projected/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-kube-api-access-b5nqc\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314376 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-config\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314390 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314419 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314448 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314474 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314502 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314538 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314575 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314600 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314629 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314659 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.314775 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.314767 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.315360 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.315177 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.315360 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.315188 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.315360 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.315245 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.315360 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.315273 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.316604 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.316576 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.316718 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.316618 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.317675 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.317550 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.318378 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.318167 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.318378 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.318310 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.318823 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.318790 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.318946 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.318922 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.319029 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.318981 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.319131 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.319092 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.319131 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.319118 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.319635 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.319618 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.319635 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.319624 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.319742 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.319681 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-config\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.319884 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.319862 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.320488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.320471 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.320980 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.320967 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.324167 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.324147 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5nqc\" (UniqueName: \"kubernetes.io/projected/4f7c6b38-04ec-4a48-b7e9-95e67a5b340e-kube-api-access-b5nqc\") pod \"prometheus-k8s-0\" (UID: \"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.467024 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.466994 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:48.765349 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.765319 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 20:53:48.765587 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:53:48.765567 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7c6b38_04ec_4a48_b7e9_95e67a5b340e.slice/crio-abfb234e95c9197d42e2f8db1e1116f64b76693fff42b1960bc5e90134711917 WatchSource:0}: Error finding container abfb234e95c9197d42e2f8db1e1116f64b76693fff42b1960bc5e90134711917: Status 404 returned error can't find the container with id abfb234e95c9197d42e2f8db1e1116f64b76693fff42b1960bc5e90134711917 Sep 09 20:53:48.869317 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:48.869289 2538 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e76bf5e-7bcf-4221-a20a-9237d557bb88" path="/var/lib/kubelet/pods/4e76bf5e-7bcf-4221-a20a-9237d557bb88/volumes" Sep 09 20:53:49.073898 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:49.073863 2538 generic.go:356] "Generic (PLEG): container finished" podID="4f7c6b38-04ec-4a48-b7e9-95e67a5b340e" containerID="066b41dd8e1de3a2e4a66bf62cabe007c52c0f04ab6c6f6bdcaf835745adc811" exitCode=0 Sep 09 20:53:49.074045 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:49.073933 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerDied","Data":"066b41dd8e1de3a2e4a66bf62cabe007c52c0f04ab6c6f6bdcaf835745adc811"} Sep 09 20:53:49.074045 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:49.073983 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerStarted","Data":"abfb234e95c9197d42e2f8db1e1116f64b76693fff42b1960bc5e90134711917"} Sep 09 20:53:50.086735 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.086703 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerStarted","Data":"d4ed01317fa26577285983fc1ea78eeb4da8e555dec1747a3985634c6b7c80fe"} Sep 09 20:53:50.087084 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.086788 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerStarted","Data":"f0dc5edc4e68dcb1a0d150635856a45352f5e8ab740cf5a700eb42a8e6d9e013"} Sep 09 20:53:50.087084 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.086815 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerStarted","Data":"fc171ffe20f51a5abe52b01c2b21a5daefe60a2b37f4f15c9f00c929c16e9882"} Sep 09 20:53:50.087084 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.086838 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerStarted","Data":"8ff17b32c9aebe47133fe103f3b66a1cf94a7d3f8aaf8dd8026a607d3086ad84"} Sep 09 20:53:50.087084 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.086861 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerStarted","Data":"299aedfbf3c3ef688978034f7dd03b5f2c1f361c000332d6214ca76fbd967f7c"} Sep 09 20:53:50.087084 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.086882 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7c6b38-04ec-4a48-b7e9-95e67a5b340e","Type":"ContainerStarted","Data":"736c7f2ce75fdd32406919b4ca749183341997c85a2bb59a0be512fd3893579e"} Sep 09 20:53:50.119549 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.119493 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.099286902 podStartE2EDuration="2.119473476s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.186574ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:50.118136143 +0000 UTC m=+371.736796392" watchObservedRunningTime="2026-09-09 20:53:50.119473476 +0000 UTC m=+371.738133711" Sep 09 20:53:50.692699 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.692669 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:50.795337 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:50.795302 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:53.467778 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:53.467747 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:53:54.412040 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:54.412005 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:54.497992 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:54.497959 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:53:54.566073 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:53:54.566043 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:03.419357 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:03.419319 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:03.517492 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:03.517444 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:05.441506 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:05.441468 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:05.543411 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:05.543377 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:05.641621 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:05.641588 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:13.362627 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:13.362589 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:13.423860 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:13.423832 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:54:48.468134 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:48.468098 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:54:48.484113 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:48.484089 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:54:49.322860 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:54:49.322832 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 20:55:00.933350 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:00.933253 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:01.027393 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:01.027359 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:27.926019 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:27.925971 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:29.591723 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:29.591685 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:31.532124 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:31.532091 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:31.636768 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:31.636733 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:33.010408 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:33.010367 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:33.126579 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:33.126544 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:33.535386 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:33.535349 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:33.658732 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:33.658691 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:35.911574 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:35.911544 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:36.124037 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:36.124007 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:38.315842 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:38.315806 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:38.421077 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:38.421021 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:43.703279 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:43.703247 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:45.802821 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:45.802784 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:45.913159 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:45.913132 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:51.382144 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:51.382107 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:51.479929 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:51.479905 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:55:53.615993 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:55:53.615939 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:00.658941 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:00.658905 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:00.798745 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:00.798716 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:07.325088 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:07.325055 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:07.416599 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:07.416554 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:07.949854 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:07.949821 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:08.163580 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.163526 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:08.277176 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.277075 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:08.423042 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.423007 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf"] Sep 09 20:56:08.426243 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.426202 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.432691 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:08.432660 2538 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-69dvb\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-69dvb\"" type="*v1.Secret" Sep 09 20:56:08.432821 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:08.432754 2538 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" Sep 09 20:56:08.433820 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.433796 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 20:56:08.434059 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.434044 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 20:56:08.434298 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.434282 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 20:56:08.462327 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.462303 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf"] Sep 09 20:56:08.488072 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.488044 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48345d2d-d88c-4551-8cf5-c4d97de63c4f-tmp\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.488218 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.488094 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.488218 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.488113 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-webhook-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.488218 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.488143 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48345d2d-d88c-4551-8cf5-c4d97de63c4f-manifests\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.488218 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.488176 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrprx\" (UniqueName: \"kubernetes.io/projected/48345d2d-d88c-4551-8cf5-c4d97de63c4f-kube-api-access-wrprx\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.568862 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.568829 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:08.589005 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.588980 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.589005 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.589007 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-webhook-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.589177 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.589026 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48345d2d-d88c-4551-8cf5-c4d97de63c4f-manifests\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.589177 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.589158 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrprx\" (UniqueName: \"kubernetes.io/projected/48345d2d-d88c-4551-8cf5-c4d97de63c4f-kube-api-access-wrprx\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.589295 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.589212 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48345d2d-d88c-4551-8cf5-c4d97de63c4f-tmp\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.589380 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.589365 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48345d2d-d88c-4551-8cf5-c4d97de63c4f-manifests\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.591455 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.591436 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48345d2d-d88c-4551-8cf5-c4d97de63c4f-tmp\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.598747 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.598724 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrprx\" (UniqueName: \"kubernetes.io/projected/48345d2d-d88c-4551-8cf5-c4d97de63c4f-kube-api-access-wrprx\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:08.678960 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.678926 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:08.772876 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.772846 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:08.870715 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.870630 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:08.971815 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:08.971778 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:09.039524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:09.039491 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:09.142972 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:09.142903 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:09.240095 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:09.240058 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:09.589871 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:09.589844 2538 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Sep 09 20:56:09.589871 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:09.589865 2538 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Sep 09 20:56:09.590342 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:09.589935 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-apiservice-cert podName:48345d2d-d88c-4551-8cf5-c4d97de63c4f nodeName:}" failed. No retries permitted until 2026-09-09 20:56:10.089914183 +0000 UTC m=+511.708574393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-apiservice-cert") pod "opendatahub-operator-controller-manager-97cf5684c-5sfkf" (UID: "48345d2d-d88c-4551-8cf5-c4d97de63c4f") : failed to sync secret cache: timed out waiting for the condition Sep 09 20:56:09.590342 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:09.589957 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-webhook-cert podName:48345d2d-d88c-4551-8cf5-c4d97de63c4f nodeName:}" failed. No retries permitted until 2026-09-09 20:56:10.089947165 +0000 UTC m=+511.708607374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-webhook-cert") pod "opendatahub-operator-controller-manager-97cf5684c-5sfkf" (UID: "48345d2d-d88c-4551-8cf5-c4d97de63c4f") : failed to sync secret cache: timed out waiting for the condition Sep 09 20:56:09.664969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:09.664946 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-69dvb\"" Sep 09 20:56:09.696268 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:09.696248 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 20:56:10.102168 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.102139 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:10.102341 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.102173 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-webhook-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:10.105139 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.105115 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:10.105139 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.105132 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48345d2d-d88c-4551-8cf5-c4d97de63c4f-webhook-cert\") pod \"opendatahub-operator-controller-manager-97cf5684c-5sfkf\" (UID: \"48345d2d-d88c-4551-8cf5-c4d97de63c4f\") " pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:10.236415 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.236373 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:10.517739 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.517694 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf"] Sep 09 20:56:10.520374 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:56:10.520335 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48345d2d_d88c_4551_8cf5_c4d97de63c4f.slice/crio-a07aabedff3c38b15d44e254140660c45b9e8c4ebeeb06558e530363a11029af WatchSource:0}: Error finding container a07aabedff3c38b15d44e254140660c45b9e8c4ebeeb06558e530363a11029af: Status 404 returned error can't find the container with id a07aabedff3c38b15d44e254140660c45b9e8c4ebeeb06558e530363a11029af Sep 09 20:56:10.522639 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.522612 2538 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 20:56:10.595011 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:10.594985 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" event={"ID":"48345d2d-d88c-4551-8cf5-c4d97de63c4f","Type":"ContainerStarted","Data":"a07aabedff3c38b15d44e254140660c45b9e8c4ebeeb06558e530363a11029af"} Sep 09 20:56:13.609393 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:13.609312 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" event={"ID":"48345d2d-d88c-4551-8cf5-c4d97de63c4f","Type":"ContainerStarted","Data":"35636f20c31b983c30b4606d31a6a362aec9d66e551303783c51e4f3d887e090"} Sep 09 20:56:14.615009 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:14.614979 2538 generic.go:356] "Generic (PLEG): container finished" podID="48345d2d-d88c-4551-8cf5-c4d97de63c4f" containerID="35636f20c31b983c30b4606d31a6a362aec9d66e551303783c51e4f3d887e090" exitCode=0 Sep 09 20:56:14.615462 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:14.615061 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" event={"ID":"48345d2d-d88c-4551-8cf5-c4d97de63c4f","Type":"ContainerDied","Data":"35636f20c31b983c30b4606d31a6a362aec9d66e551303783c51e4f3d887e090"} Sep 09 20:56:15.621065 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:15.621032 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" event={"ID":"48345d2d-d88c-4551-8cf5-c4d97de63c4f","Type":"ContainerStarted","Data":"fbb636733e5632d7bf7bfee6bc5123076015ef7c453011ec7efe1ef8d72c2aeb"} Sep 09 20:56:15.621454 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:15.621170 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:15.645460 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:15.645366 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" podStartSLOduration=3.026079097 podStartE2EDuration="7.645351433s" totalImagesPullingTime="3.559172082s" totalInitContainerRuntime="1.060100254s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:15.644077523 +0000 UTC m=+517.262737754" watchObservedRunningTime="2026-09-09 20:56:15.645351433 +0000 UTC m=+517.264011664" Sep 09 20:56:19.936283 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:19.936244 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:20.044844 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:20.044809 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:26.626330 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:26.626298 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-97cf5684c-5sfkf" Sep 09 20:56:28.299809 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:28.299775 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:28.438419 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:28.438386 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:28.724870 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:28.724764 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:29.094542 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:29.094507 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:30.316065 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:30.315957 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:30.409529 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:30.409487 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:31.474897 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:31.474861 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:31.945463 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:31.945428 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:32.051028 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:32.050975 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:35.190787 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:35.190751 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:35.292916 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:35.292886 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:35.480114 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:35.480040 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:35.625541 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:35.625509 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:35.722733 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:35.722679 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:39.316458 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:39.316425 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:39.415270 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:39.415210 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:39.515201 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:39.515164 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:39.835640 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:39.835605 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:39.915854 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:39.915826 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:40.194798 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:40.194704 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:47.966485 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.966445 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk"] Sep 09 20:56:47.969411 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.969393 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:47.973787 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.973765 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4lv2q\"" Sep 09 20:56:47.974134 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.974113 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 09 20:56:47.974251 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.974187 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 09 20:56:47.974870 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.974855 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 09 20:56:47.976059 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.976038 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:56:47.976447 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:47.976432 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 09 20:56:48.021977 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.021946 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk"] Sep 09 20:56:48.105372 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.105346 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2f801e-0b13-4861-949e-3c2ef230b977-cert\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.105529 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.105418 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2vrw\" (UniqueName: \"kubernetes.io/projected/aa2f801e-0b13-4861-949e-3c2ef230b977-kube-api-access-p2vrw\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.105529 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.105445 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa2f801e-0b13-4861-949e-3c2ef230b977-manager-config\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.105529 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.105464 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa2f801e-0b13-4861-949e-3c2ef230b977-metrics-cert\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.206842 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.206812 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2f801e-0b13-4861-949e-3c2ef230b977-cert\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.207012 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.206880 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2vrw\" (UniqueName: \"kubernetes.io/projected/aa2f801e-0b13-4861-949e-3c2ef230b977-kube-api-access-p2vrw\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.207012 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.206905 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa2f801e-0b13-4861-949e-3c2ef230b977-manager-config\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.207012 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.206923 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa2f801e-0b13-4861-949e-3c2ef230b977-metrics-cert\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.207610 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.207587 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa2f801e-0b13-4861-949e-3c2ef230b977-manager-config\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.209381 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.209354 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2f801e-0b13-4861-949e-3c2ef230b977-cert\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.209468 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.209454 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa2f801e-0b13-4861-949e-3c2ef230b977-metrics-cert\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.229956 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.229894 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2vrw\" (UniqueName: \"kubernetes.io/projected/aa2f801e-0b13-4861-949e-3c2ef230b977-kube-api-access-p2vrw\") pod \"lws-controller-manager-7596c9fb79-ffjfk\" (UID: \"aa2f801e-0b13-4861-949e-3c2ef230b977\") " pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.279511 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.279487 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:48.589735 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.589701 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk"] Sep 09 20:56:48.594888 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:56:48.594865 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2f801e_0b13_4861_949e_3c2ef230b977.slice/crio-6a18fb8abf5f8d6f0a7f41fb148e4eb13ba70688edc2f0b8dc54d3a96a478fe7 WatchSource:0}: Error finding container 6a18fb8abf5f8d6f0a7f41fb148e4eb13ba70688edc2f0b8dc54d3a96a478fe7: Status 404 returned error can't find the container with id 6a18fb8abf5f8d6f0a7f41fb148e4eb13ba70688edc2f0b8dc54d3a96a478fe7 Sep 09 20:56:48.747803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:48.747775 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" event={"ID":"aa2f801e-0b13-4861-949e-3c2ef230b977","Type":"ContainerStarted","Data":"6a18fb8abf5f8d6f0a7f41fb148e4eb13ba70688edc2f0b8dc54d3a96a478fe7"} Sep 09 20:56:51.938827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:51.938783 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:52.104150 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.104118 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt"] Sep 09 20:56:52.107429 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.107413 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.129161 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.129141 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2qxsv\"" Sep 09 20:56:52.146310 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.146287 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 09 20:56:52.151218 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.151195 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt"] Sep 09 20:56:52.178353 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.178330 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr"] Sep 09 20:56:52.181470 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.181455 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:52.184045 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.184026 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 09 20:56:52.185494 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.185476 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rrcxd\"" Sep 09 20:56:52.204719 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.204697 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr"] Sep 09 20:56:52.240670 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.240645 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5-cert\") pod \"kserve-controller-manager-d6f4b65b8-tj8pt\" (UID: \"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.240801 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.240680 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj4fh\" (UniqueName: \"kubernetes.io/projected/cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5-kube-api-access-sj4fh\") pod \"kserve-controller-manager-d6f4b65b8-tj8pt\" (UID: \"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.341569 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.341538 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj4fh\" (UniqueName: \"kubernetes.io/projected/cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5-kube-api-access-sj4fh\") pod \"kserve-controller-manager-d6f4b65b8-tj8pt\" (UID: \"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.341721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.341576 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85pzg\" (UniqueName: \"kubernetes.io/projected/bbdc23d5-8f42-4b6b-9ede-412e927b145d-kube-api-access-85pzg\") pod \"llmisvc-controller-manager-777c6bd7d5-krvtr\" (UID: \"bbdc23d5-8f42-4b6b-9ede-412e927b145d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:52.341721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.341680 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc23d5-8f42-4b6b-9ede-412e927b145d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-krvtr\" (UID: \"bbdc23d5-8f42-4b6b-9ede-412e927b145d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:52.341793 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.341722 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5-cert\") pod \"kserve-controller-manager-d6f4b65b8-tj8pt\" (UID: \"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.344399 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.344375 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5-cert\") pod \"kserve-controller-manager-d6f4b65b8-tj8pt\" (UID: \"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.355026 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.354967 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj4fh\" (UniqueName: \"kubernetes.io/projected/cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5-kube-api-access-sj4fh\") pod \"kserve-controller-manager-d6f4b65b8-tj8pt\" (UID: \"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.416774 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.416734 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:52.442253 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.442207 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85pzg\" (UniqueName: \"kubernetes.io/projected/bbdc23d5-8f42-4b6b-9ede-412e927b145d-kube-api-access-85pzg\") pod \"llmisvc-controller-manager-777c6bd7d5-krvtr\" (UID: \"bbdc23d5-8f42-4b6b-9ede-412e927b145d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:52.442375 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.442358 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc23d5-8f42-4b6b-9ede-412e927b145d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-krvtr\" (UID: \"bbdc23d5-8f42-4b6b-9ede-412e927b145d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:52.442501 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:52.442481 2538 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Sep 09 20:56:52.442561 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:56:52.442551 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbdc23d5-8f42-4b6b-9ede-412e927b145d-cert podName:bbdc23d5-8f42-4b6b-9ede-412e927b145d nodeName:}" failed. No retries permitted until 2026-09-09 20:56:52.94253109 +0000 UTC m=+554.561191320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bbdc23d5-8f42-4b6b-9ede-412e927b145d-cert") pod "llmisvc-controller-manager-777c6bd7d5-krvtr" (UID: "bbdc23d5-8f42-4b6b-9ede-412e927b145d") : secret "llmisvc-webhook-server-cert" not found Sep 09 20:56:52.455802 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.455770 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85pzg\" (UniqueName: \"kubernetes.io/projected/bbdc23d5-8f42-4b6b-9ede-412e927b145d-kube-api-access-85pzg\") pod \"llmisvc-controller-manager-777c6bd7d5-krvtr\" (UID: \"bbdc23d5-8f42-4b6b-9ede-412e927b145d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:52.713107 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:56:52.713076 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4c6ee2_f597_4a8f_a5c9_32a073e2bbb5.slice/crio-905c91d912de88ca9fa95ed18c59e64d46c4642a8628601e4f6993fc8853221f WatchSource:0}: Error finding container 905c91d912de88ca9fa95ed18c59e64d46c4642a8628601e4f6993fc8853221f: Status 404 returned error can't find the container with id 905c91d912de88ca9fa95ed18c59e64d46c4642a8628601e4f6993fc8853221f Sep 09 20:56:52.716994 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.716971 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt"] Sep 09 20:56:52.769866 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.769836 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" event={"ID":"aa2f801e-0b13-4861-949e-3c2ef230b977","Type":"ContainerStarted","Data":"ad61c37f2117adae2a213365357418f44e389a1d6e06267a0c5a8c931b3ef112"} Sep 09 20:56:52.770035 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.769916 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:56:52.771107 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.771088 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" event={"ID":"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5","Type":"ContainerStarted","Data":"905c91d912de88ca9fa95ed18c59e64d46c4642a8628601e4f6993fc8853221f"} Sep 09 20:56:52.846830 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.846790 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" podStartSLOduration=2.742173575 podStartE2EDuration="5.84677644s" totalImagesPullingTime="3.104602865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:52.845809649 +0000 UTC m=+554.464469881" watchObservedRunningTime="2026-09-09 20:56:52.84677644 +0000 UTC m=+554.465436672" Sep 09 20:56:52.946502 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.946465 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc23d5-8f42-4b6b-9ede-412e927b145d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-krvtr\" (UID: \"bbdc23d5-8f42-4b6b-9ede-412e927b145d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:52.949292 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:52.949262 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc23d5-8f42-4b6b-9ede-412e927b145d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-krvtr\" (UID: \"bbdc23d5-8f42-4b6b-9ede-412e927b145d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:53.091322 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:53.091283 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:53.384150 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:53.384050 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr"] Sep 09 20:56:53.386526 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:56:53.386493 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbbdc23d5_8f42_4b6b_9ede_412e927b145d.slice/crio-3e7fbcdff0b5bbc6cc2a38db58e317947268af8e32a46cc0d66327f5a49719c5 WatchSource:0}: Error finding container 3e7fbcdff0b5bbc6cc2a38db58e317947268af8e32a46cc0d66327f5a49719c5: Status 404 returned error can't find the container with id 3e7fbcdff0b5bbc6cc2a38db58e317947268af8e32a46cc0d66327f5a49719c5 Sep 09 20:56:53.778735 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:53.778647 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" event={"ID":"bbdc23d5-8f42-4b6b-9ede-412e927b145d","Type":"ContainerStarted","Data":"3e7fbcdff0b5bbc6cc2a38db58e317947268af8e32a46cc0d66327f5a49719c5"} Sep 09 20:56:53.857683 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:53.857652 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:56:56.794697 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:56.794663 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" event={"ID":"bbdc23d5-8f42-4b6b-9ede-412e927b145d","Type":"ContainerStarted","Data":"a101a1f34abe5ef7886b69f92b8fc8a7deb306627603c9c4001a504058a4fcbc"} Sep 09 20:56:56.795050 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:56.794795 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:56:56.796133 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:56.796113 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" event={"ID":"cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5","Type":"ContainerStarted","Data":"fb1d607396dc90c6d869ac6a9539db757e19d01c464f7396d62d6a07e856729f"} Sep 09 20:56:56.796272 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:56.796260 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:56:56.819693 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:56.819658 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" podStartSLOduration=1.564429483 podStartE2EDuration="4.819644662s" totalImagesPullingTime="3.255215179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:56.818467276 +0000 UTC m=+558.437127507" watchObservedRunningTime="2026-09-09 20:56:56.819644662 +0000 UTC m=+558.438304895" Sep 09 20:56:56.848214 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:56:56.848136 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" podStartSLOduration=1.699077257 podStartE2EDuration="4.848121108s" totalImagesPullingTime="3.149043851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:56.846610655 +0000 UTC m=+558.465270888" watchObservedRunningTime="2026-09-09 20:56:56.848121108 +0000 UTC m=+558.466781339" Sep 09 20:57:03.781648 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:03.781617 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7596c9fb79-ffjfk" Sep 09 20:57:06.479408 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:06.479370 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:06.549793 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:06.549759 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:11.352115 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:11.352084 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:11.420071 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:11.420032 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:22.355080 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:22.355043 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:22.455618 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:22.455591 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:22.506675 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:22.506637 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:27.802364 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:27.802333 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-krvtr" Sep 09 20:57:27.805304 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:27.805288 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tj8pt" Sep 09 20:57:40.331158 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:40.331124 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:40.420416 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:40.420385 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:56.657720 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:56.657685 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:57:57.513825 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:57:57.513790 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:00.482633 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:00.482546 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:00.792947 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:00.792871 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:01.146381 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:01.146340 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:01.342546 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:01.342511 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:01.523912 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:01.523834 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:03.166041 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:03.166005 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:04.605491 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.605456 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8"] Sep 09 20:58:04.608590 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.608573 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:04.611543 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.611521 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 20:58:04.611543 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.611538 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 20:58:04.611686 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.611646 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zs8dt\"" Sep 09 20:58:04.632308 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.632284 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-584cp\" (UniqueName: \"kubernetes.io/projected/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-kube-api-access-584cp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:04.632438 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.632336 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:04.633933 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.633911 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8"] Sep 09 20:58:04.733313 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.733281 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-584cp\" (UniqueName: \"kubernetes.io/projected/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-kube-api-access-584cp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:04.733449 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.733324 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:04.733659 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.733640 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:04.746876 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.746849 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-584cp\" (UniqueName: \"kubernetes.io/projected/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-kube-api-access-584cp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:04.918003 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:04.917915 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:05.046423 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:05.046393 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:05.200869 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:05.200791 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:05.229252 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:58:05.229196 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85bb4ce1_fa44_462e_89dd_31fcfc00f7ec.slice/crio-5abf261e7732c1a6726164fd29ecb9acb96c772c5df9f2755d4d149569d30467 WatchSource:0}: Error finding container 5abf261e7732c1a6726164fd29ecb9acb96c772c5df9f2755d4d149569d30467: Status 404 returned error can't find the container with id 5abf261e7732c1a6726164fd29ecb9acb96c772c5df9f2755d4d149569d30467 Sep 09 20:58:05.231283 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:05.231257 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8"] Sep 09 20:58:05.882837 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:05.882805 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:06.091849 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:06.091819 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" event={"ID":"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec","Type":"ContainerStarted","Data":"5abf261e7732c1a6726164fd29ecb9acb96c772c5df9f2755d4d149569d30467"} Sep 09 20:58:07.375378 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:07.375333 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:08.466097 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:08.466055 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:09.278908 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.278871 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd"] Sep 09 20:58:09.282960 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.282926 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" Sep 09 20:58:09.293958 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.293928 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 09 20:58:09.295210 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.295187 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-fhc6x\"" Sep 09 20:58:09.315753 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.315730 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd"] Sep 09 20:58:09.375301 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.375271 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6tvf\" (UniqueName: \"kubernetes.io/projected/53095058-cb0c-4715-a856-020743bb74de-kube-api-access-q6tvf\") pod \"dns-operator-controller-manager-7b84cd7c9c-274nd\" (UID: \"53095058-cb0c-4715-a856-020743bb74de\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" Sep 09 20:58:09.476124 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.476101 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6tvf\" (UniqueName: \"kubernetes.io/projected/53095058-cb0c-4715-a856-020743bb74de-kube-api-access-q6tvf\") pod \"dns-operator-controller-manager-7b84cd7c9c-274nd\" (UID: \"53095058-cb0c-4715-a856-020743bb74de\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" Sep 09 20:58:09.490845 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.490824 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6tvf\" (UniqueName: \"kubernetes.io/projected/53095058-cb0c-4715-a856-020743bb74de-kube-api-access-q6tvf\") pod \"dns-operator-controller-manager-7b84cd7c9c-274nd\" (UID: \"53095058-cb0c-4715-a856-020743bb74de\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" Sep 09 20:58:09.596022 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.595993 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" Sep 09 20:58:09.886768 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:09.886695 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd"] Sep 09 20:58:09.891518 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:58:09.891488 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53095058_cb0c_4715_a856_020743bb74de.slice/crio-2ec00f5079a99adbb32a13c034b219877b8f928d1501494ec6ffecd8e989a2af WatchSource:0}: Error finding container 2ec00f5079a99adbb32a13c034b219877b8f928d1501494ec6ffecd8e989a2af: Status 404 returned error can't find the container with id 2ec00f5079a99adbb32a13c034b219877b8f928d1501494ec6ffecd8e989a2af Sep 09 20:58:10.121026 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:10.120994 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" event={"ID":"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec","Type":"ContainerStarted","Data":"c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f"} Sep 09 20:58:10.121250 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:10.121114 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:10.122378 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:10.122352 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" event={"ID":"53095058-cb0c-4715-a856-020743bb74de","Type":"ContainerStarted","Data":"2ec00f5079a99adbb32a13c034b219877b8f928d1501494ec6ffecd8e989a2af"} Sep 09 20:58:10.154855 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:10.154768 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" podStartSLOduration=1.9467750860000002 podStartE2EDuration="6.154753017s" totalImagesPullingTime="4.207977931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:10.152665561 +0000 UTC m=+631.771325794" watchObservedRunningTime="2026-09-09 20:58:10.154753017 +0000 UTC m=+631.773413250" Sep 09 20:58:10.553777 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:10.553744 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:10.686921 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:10.686884 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:10.959790 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:10.959703 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:11.065395 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:11.065353 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:12.135573 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:12.135538 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" event={"ID":"53095058-cb0c-4715-a856-020743bb74de","Type":"ContainerStarted","Data":"feedc1aeba11e76da64737faf6f8a817776762b897cde7ea41ab7364ac399269"} Sep 09 20:58:12.135962 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:12.135711 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" Sep 09 20:58:12.162827 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:12.162772 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" podStartSLOduration=1.337696496 podStartE2EDuration="3.16275557s" totalImagesPullingTime="1.825059074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:12.160545252 +0000 UTC m=+633.779205483" watchObservedRunningTime="2026-09-09 20:58:12.16275557 +0000 UTC m=+633.781415802" Sep 09 20:58:12.491852 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:12.491784 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:12.667537 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:12.667501 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:13.769656 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:13.769621 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:15.334949 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:15.334915 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:15.522500 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:15.522465 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:15.809414 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:15.809372 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:16.228287 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:16.228194 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:21.130639 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:21.130607 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:22.723804 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:22.723764 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:23.118078 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:23.118047 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:23.143810 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:23.143787 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-274nd" Sep 09 20:58:24.250946 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.250913 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8"] Sep 09 20:58:24.251367 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.251077 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" podUID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" containerName="manager" containerID="cri-o://c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f" gracePeriod=2 Sep 09 20:58:24.276415 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.276356 2538 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8"] Sep 09 20:58:24.334897 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.334868 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd"] Sep 09 20:58:24.335267 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.335245 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" containerName="manager" Sep 09 20:58:24.335416 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.335288 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" containerName="manager" Sep 09 20:58:24.335416 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.335379 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" containerName="manager" Sep 09 20:58:24.338395 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.338377 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.365413 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.365385 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd"] Sep 09 20:58:24.403604 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.403575 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgc78\" (UniqueName: \"kubernetes.io/projected/b34a1476-2a55-4625-8aac-db486fb76061-kube-api-access-qgc78\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j4vmd\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.403750 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.403631 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34a1476-2a55-4625-8aac-db486fb76061-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j4vmd\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.441976 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:58:24.441932 2538 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:24.488607 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.488581 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:24.491849 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:58:24.491827 2538 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8n2f8\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:24.504308 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.504263 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgc78\" (UniqueName: \"kubernetes.io/projected/b34a1476-2a55-4625-8aac-db486fb76061-kube-api-access-qgc78\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j4vmd\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.504376 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.504315 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34a1476-2a55-4625-8aac-db486fb76061-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j4vmd\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.504609 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.504593 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34a1476-2a55-4625-8aac-db486fb76061-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j4vmd\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.518712 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.518685 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgc78\" (UniqueName: \"kubernetes.io/projected/b34a1476-2a55-4625-8aac-db486fb76061-kube-api-access-qgc78\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j4vmd\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.604863 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.604834 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-584cp\" (UniqueName: \"kubernetes.io/projected/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-kube-api-access-584cp\") pod \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " Sep 09 20:58:24.605008 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.604878 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-extensions-socket-volume\") pod \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\" (UID: \"85bb4ce1-fa44-462e-89dd-31fcfc00f7ec\") " Sep 09 20:58:24.605395 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.605371 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" (UID: "85bb4ce1-fa44-462e-89dd-31fcfc00f7ec"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:58:24.607166 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.607145 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-kube-api-access-584cp" (OuterVolumeSpecName: "kube-api-access-584cp") pod "85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" (UID: "85bb4ce1-fa44-462e-89dd-31fcfc00f7ec"). InnerVolumeSpecName "kube-api-access-584cp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:58:24.649939 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.649911 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:24.705989 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.705958 2538 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-584cp\" (UniqueName: \"kubernetes.io/projected/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-kube-api-access-584cp\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:58:24.706092 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.705995 2538 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec-extensions-socket-volume\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:58:24.869713 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.869682 2538 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" path="/var/lib/kubelet/pods/85bb4ce1-fa44-462e-89dd-31fcfc00f7ec/volumes" Sep 09 20:58:24.950793 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:24.950759 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd"] Sep 09 20:58:25.080244 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.080194 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:25.191922 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.191820 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" event={"ID":"b34a1476-2a55-4625-8aac-db486fb76061","Type":"ContainerStarted","Data":"5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d"} Sep 09 20:58:25.191922 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.191887 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" event={"ID":"b34a1476-2a55-4625-8aac-db486fb76061","Type":"ContainerStarted","Data":"6392095131da24be68b8670da872e6a5adaec7a2cbf35027ee23bedc2c991213"} Sep 09 20:58:25.191922 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.191912 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:25.193266 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.193241 2538 generic.go:356] "Generic (PLEG): container finished" podID="85bb4ce1-fa44-462e-89dd-31fcfc00f7ec" containerID="c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f" exitCode=0 Sep 09 20:58:25.193383 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.193287 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8n2f8" Sep 09 20:58:25.193383 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.193290 2538 scope.go:117] "RemoveContainer" containerID="c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f" Sep 09 20:58:25.201418 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.201399 2538 scope.go:117] "RemoveContainer" containerID="c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f" Sep 09 20:58:25.201682 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:58:25.201663 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f\": container with ID starting with c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f not found: ID does not exist" containerID="c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f" Sep 09 20:58:25.201746 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.201694 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f"} err="failed to get container status \"c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f\": rpc error: code = NotFound desc = could not find container \"c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f\": container with ID starting with c87e794dc551111f7eb0106d8045484fa4837469a4b73edb0cb6b4481c185a4f not found: ID does not exist" Sep 09 20:58:25.236911 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.236868 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" podStartSLOduration=1.236852571 podStartE2EDuration="1.236852571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:25.233542044 +0000 UTC m=+646.852202276" watchObservedRunningTime="2026-09-09 20:58:25.236852571 +0000 UTC m=+646.855512803" Sep 09 20:58:25.248489 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:25.248460 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:26.429471 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.429433 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r"] Sep 09 20:58:26.434123 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.434102 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:26.453952 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.453927 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r"] Sep 09 20:58:26.523305 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.523278 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sgsh\" (UniqueName: \"kubernetes.io/projected/7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1-kube-api-access-6sgsh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bpm7r\" (UID: \"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:26.523426 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.523318 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bpm7r\" (UID: \"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:26.624219 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.624193 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sgsh\" (UniqueName: \"kubernetes.io/projected/7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1-kube-api-access-6sgsh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bpm7r\" (UID: \"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:26.624350 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.624249 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bpm7r\" (UID: \"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:26.624585 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.624560 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bpm7r\" (UID: \"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:26.633514 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.633484 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sgsh\" (UniqueName: \"kubernetes.io/projected/7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1-kube-api-access-6sgsh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bpm7r\" (UID: \"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:26.744887 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:26.744826 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:27.022167 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:27.022090 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r"] Sep 09 20:58:27.025169 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:58:27.025144 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cefd40f_ce8f_4d50_9be1_dc1ff25ae0a1.slice/crio-4ce722bd5e21cff0860fcd0d0501e8fe54c962be35669abb32a1fd5a7db8fb35 WatchSource:0}: Error finding container 4ce722bd5e21cff0860fcd0d0501e8fe54c962be35669abb32a1fd5a7db8fb35: Status 404 returned error can't find the container with id 4ce722bd5e21cff0860fcd0d0501e8fe54c962be35669abb32a1fd5a7db8fb35 Sep 09 20:58:27.206063 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:27.206033 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" event={"ID":"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1","Type":"ContainerStarted","Data":"5f7fd46a72cfa0ce5a85b24fbe73a4230a2b8359bda0ab0f3667846fdb519eb2"} Sep 09 20:58:27.206213 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:27.206080 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" event={"ID":"7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1","Type":"ContainerStarted","Data":"4ce722bd5e21cff0860fcd0d0501e8fe54c962be35669abb32a1fd5a7db8fb35"} Sep 09 20:58:27.206213 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:27.206111 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:27.232899 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:27.232854 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" podStartSLOduration=1.232840591 podStartE2EDuration="1.232840591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:27.232279766 +0000 UTC m=+648.850939998" watchObservedRunningTime="2026-09-09 20:58:27.232840591 +0000 UTC m=+648.851500873" Sep 09 20:58:30.677256 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:30.677204 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:30.778799 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:30.778759 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:31.285265 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:31.285208 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:31.390992 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:31.390958 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:35.809969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:35.809927 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:35.909758 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:35.909707 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:36.201247 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:36.201162 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:36.206501 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:36.206465 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:36.307614 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:36.307578 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:36.616539 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:36.616505 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:36.689835 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:36.689807 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:38.213368 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.213343 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bpm7r" Sep 09 20:58:38.288756 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.288728 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd"] Sep 09 20:58:38.288999 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.288967 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" podUID="b34a1476-2a55-4625-8aac-db486fb76061" containerName="manager" containerID="cri-o://5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d" gracePeriod=10 Sep 09 20:58:38.528928 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.528906 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:38.621574 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.621547 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34a1476-2a55-4625-8aac-db486fb76061-extensions-socket-volume\") pod \"b34a1476-2a55-4625-8aac-db486fb76061\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " Sep 09 20:58:38.621726 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.621595 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgc78\" (UniqueName: \"kubernetes.io/projected/b34a1476-2a55-4625-8aac-db486fb76061-kube-api-access-qgc78\") pod \"b34a1476-2a55-4625-8aac-db486fb76061\" (UID: \"b34a1476-2a55-4625-8aac-db486fb76061\") " Sep 09 20:58:38.621948 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.621923 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34a1476-2a55-4625-8aac-db486fb76061-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b34a1476-2a55-4625-8aac-db486fb76061" (UID: "b34a1476-2a55-4625-8aac-db486fb76061"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:58:38.623651 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.623626 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34a1476-2a55-4625-8aac-db486fb76061-kube-api-access-qgc78" (OuterVolumeSpecName: "kube-api-access-qgc78") pod "b34a1476-2a55-4625-8aac-db486fb76061" (UID: "b34a1476-2a55-4625-8aac-db486fb76061"). InnerVolumeSpecName "kube-api-access-qgc78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:58:38.722832 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.722804 2538 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34a1476-2a55-4625-8aac-db486fb76061-extensions-socket-volume\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:58:38.722832 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:38.722828 2538 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgc78\" (UniqueName: \"kubernetes.io/projected/b34a1476-2a55-4625-8aac-db486fb76061-kube-api-access-qgc78\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:58:39.260886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.260799 2538 generic.go:356] "Generic (PLEG): container finished" podID="b34a1476-2a55-4625-8aac-db486fb76061" containerID="5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d" exitCode=0 Sep 09 20:58:39.260886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.260845 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" Sep 09 20:58:39.260886 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.260862 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" event={"ID":"b34a1476-2a55-4625-8aac-db486fb76061","Type":"ContainerDied","Data":"5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d"} Sep 09 20:58:39.261392 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.260915 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd" event={"ID":"b34a1476-2a55-4625-8aac-db486fb76061","Type":"ContainerDied","Data":"6392095131da24be68b8670da872e6a5adaec7a2cbf35027ee23bedc2c991213"} Sep 09 20:58:39.261392 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.260944 2538 scope.go:117] "RemoveContainer" containerID="5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d" Sep 09 20:58:39.274781 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.274762 2538 scope.go:117] "RemoveContainer" containerID="5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d" Sep 09 20:58:39.275040 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:58:39.275022 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d\": container with ID starting with 5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d not found: ID does not exist" containerID="5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d" Sep 09 20:58:39.275085 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.275048 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d"} err="failed to get container status \"5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d\": rpc error: code = NotFound desc = could not find container \"5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d\": container with ID starting with 5a75210299197daa14f9a66ae11d48e3690af24c692854ea8827957293fcc86d not found: ID does not exist" Sep 09 20:58:39.282740 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.282719 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd"] Sep 09 20:58:39.289529 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:39.289506 2538 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j4vmd"] Sep 09 20:58:40.870592 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:40.870560 2538 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34a1476-2a55-4625-8aac-db486fb76061" path="/var/lib/kubelet/pods/b34a1476-2a55-4625-8aac-db486fb76061/volumes" Sep 09 20:58:43.230550 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:43.230511 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:43.333051 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:43.333023 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:45.717769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:45.717730 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:45.801987 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:45.801959 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:54.687867 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:54.687831 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:55.184235 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:55.184202 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:56.901043 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.901011 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:56.952466 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.952431 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5tzzp"] Sep 09 20:58:56.952754 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.952741 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34a1476-2a55-4625-8aac-db486fb76061" containerName="manager" Sep 09 20:58:56.952803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.952755 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34a1476-2a55-4625-8aac-db486fb76061" containerName="manager" Sep 09 20:58:56.952838 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.952803 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34a1476-2a55-4625-8aac-db486fb76061" containerName="manager" Sep 09 20:58:56.955991 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.955976 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:56.959012 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.958993 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fc772\"" Sep 09 20:58:56.959364 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.959350 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 09 20:58:56.968481 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.968458 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be4f3ca2-8263-4bf7-8186-73698c800a3f-config-file\") pod \"limitador-limitador-8588749977-5tzzp\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:56.968596 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.968503 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tksxf\" (UniqueName: \"kubernetes.io/projected/be4f3ca2-8263-4bf7-8186-73698c800a3f-kube-api-access-tksxf\") pod \"limitador-limitador-8588749977-5tzzp\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:56.976643 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:56.976623 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5tzzp"] Sep 09 20:58:57.037080 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.037045 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5tzzp"] Sep 09 20:58:57.052803 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.052773 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:57.069916 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.069879 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be4f3ca2-8263-4bf7-8186-73698c800a3f-config-file\") pod \"limitador-limitador-8588749977-5tzzp\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:57.070049 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.069923 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tksxf\" (UniqueName: \"kubernetes.io/projected/be4f3ca2-8263-4bf7-8186-73698c800a3f-kube-api-access-tksxf\") pod \"limitador-limitador-8588749977-5tzzp\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:57.070518 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.070495 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be4f3ca2-8263-4bf7-8186-73698c800a3f-config-file\") pod \"limitador-limitador-8588749977-5tzzp\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:57.081350 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.081326 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tksxf\" (UniqueName: \"kubernetes.io/projected/be4f3ca2-8263-4bf7-8186-73698c800a3f-kube-api-access-tksxf\") pod \"limitador-limitador-8588749977-5tzzp\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:57.149181 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.149138 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:57.266529 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.266451 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:57.545337 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.545269 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5tzzp"] Sep 09 20:58:57.548469 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:58:57.548443 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4f3ca2_8263_4bf7_8186_73698c800a3f.slice/crio-0a85593d11e5d1464f914a546d7838fde1bfbe20ff7daa35bb8eb7a2b846f01a WatchSource:0}: Error finding container 0a85593d11e5d1464f914a546d7838fde1bfbe20ff7daa35bb8eb7a2b846f01a: Status 404 returned error can't find the container with id 0a85593d11e5d1464f914a546d7838fde1bfbe20ff7daa35bb8eb7a2b846f01a Sep 09 20:58:57.799953 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.799922 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:57.888352 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:57.888322 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:58.349138 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:58.349106 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" event={"ID":"be4f3ca2-8263-4bf7-8186-73698c800a3f","Type":"ContainerStarted","Data":"0a85593d11e5d1464f914a546d7838fde1bfbe20ff7daa35bb8eb7a2b846f01a"} Sep 09 20:58:59.070247 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:59.070185 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:59.171332 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:59.171303 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:58:59.360863 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:59.360785 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" event={"ID":"be4f3ca2-8263-4bf7-8186-73698c800a3f","Type":"ContainerStarted","Data":"6bb2cf5437ef5a2939a8936aa5a04d1ab97f11e6d3dad4fe9817e726e7f9e484"} Sep 09 20:58:59.361201 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:59.360868 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:58:59.377112 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:58:59.377070 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" podStartSLOduration=2.15196851 podStartE2EDuration="3.377055861s" totalImagesPullingTime="1.225087351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:59.37660383 +0000 UTC m=+680.995264063" watchObservedRunningTime="2026-09-09 20:58:59.377055861 +0000 UTC m=+680.995716093" Sep 09 20:59:10.367188 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:10.367153 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:59:11.927048 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:11.927009 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5tzzp"] Sep 09 20:59:11.927615 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:11.927262 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" podUID="be4f3ca2-8263-4bf7-8186-73698c800a3f" containerName="limitador" containerID="cri-o://6bb2cf5437ef5a2939a8936aa5a04d1ab97f11e6d3dad4fe9817e726e7f9e484" gracePeriod=30 Sep 09 20:59:12.419449 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.419414 2538 generic.go:356] "Generic (PLEG): container finished" podID="be4f3ca2-8263-4bf7-8186-73698c800a3f" containerID="6bb2cf5437ef5a2939a8936aa5a04d1ab97f11e6d3dad4fe9817e726e7f9e484" exitCode=0 Sep 09 20:59:12.419595 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.419514 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" event={"ID":"be4f3ca2-8263-4bf7-8186-73698c800a3f","Type":"ContainerDied","Data":"6bb2cf5437ef5a2939a8936aa5a04d1ab97f11e6d3dad4fe9817e726e7f9e484"} Sep 09 20:59:12.471478 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.471454 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:59:12.601564 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.601537 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be4f3ca2-8263-4bf7-8186-73698c800a3f-config-file\") pod \"be4f3ca2-8263-4bf7-8186-73698c800a3f\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " Sep 09 20:59:12.601716 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.601658 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tksxf\" (UniqueName: \"kubernetes.io/projected/be4f3ca2-8263-4bf7-8186-73698c800a3f-kube-api-access-tksxf\") pod \"be4f3ca2-8263-4bf7-8186-73698c800a3f\" (UID: \"be4f3ca2-8263-4bf7-8186-73698c800a3f\") " Sep 09 20:59:12.601929 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.601906 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be4f3ca2-8263-4bf7-8186-73698c800a3f-config-file" (OuterVolumeSpecName: "config-file") pod "be4f3ca2-8263-4bf7-8186-73698c800a3f" (UID: "be4f3ca2-8263-4bf7-8186-73698c800a3f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:59:12.603901 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.603879 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be4f3ca2-8263-4bf7-8186-73698c800a3f-kube-api-access-tksxf" (OuterVolumeSpecName: "kube-api-access-tksxf") pod "be4f3ca2-8263-4bf7-8186-73698c800a3f" (UID: "be4f3ca2-8263-4bf7-8186-73698c800a3f"). InnerVolumeSpecName "kube-api-access-tksxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:59:12.702727 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.702684 2538 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tksxf\" (UniqueName: \"kubernetes.io/projected/be4f3ca2-8263-4bf7-8186-73698c800a3f-kube-api-access-tksxf\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:59:12.702727 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.702722 2538 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be4f3ca2-8263-4bf7-8186-73698c800a3f-config-file\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:59:12.907151 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:12.907124 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:13.007420 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.007390 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:13.348377 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.348344 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf"] Sep 09 20:59:13.348680 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.348669 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4f3ca2-8263-4bf7-8186-73698c800a3f" containerName="limitador" Sep 09 20:59:13.348733 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.348682 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4f3ca2-8263-4bf7-8186-73698c800a3f" containerName="limitador" Sep 09 20:59:13.348769 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.348742 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4f3ca2-8263-4bf7-8186-73698c800a3f" containerName="limitador" Sep 09 20:59:13.351721 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.351705 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.356302 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.356276 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 09 20:59:13.356451 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.356433 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lhz5m\"" Sep 09 20:59:13.356524 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.356507 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 09 20:59:13.356590 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.356577 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 09 20:59:13.374590 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.374566 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf"] Sep 09 20:59:13.408135 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.408104 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d2w7\" (UniqueName: \"kubernetes.io/projected/776b6475-3903-46b3-b7dc-e7987ea5dc89-kube-api-access-4d2w7\") pod \"postgres-69fb4594cf-9dnnf\" (UID: \"776b6475-3903-46b3-b7dc-e7987ea5dc89\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.408283 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.408194 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/776b6475-3903-46b3-b7dc-e7987ea5dc89-data\") pod \"postgres-69fb4594cf-9dnnf\" (UID: \"776b6475-3903-46b3-b7dc-e7987ea5dc89\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.418969 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.418936 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:13.424577 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.424561 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" Sep 09 20:59:13.424663 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.424593 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5tzzp" event={"ID":"be4f3ca2-8263-4bf7-8186-73698c800a3f","Type":"ContainerDied","Data":"0a85593d11e5d1464f914a546d7838fde1bfbe20ff7daa35bb8eb7a2b846f01a"} Sep 09 20:59:13.424663 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.424647 2538 scope.go:117] "RemoveContainer" containerID="6bb2cf5437ef5a2939a8936aa5a04d1ab97f11e6d3dad4fe9817e726e7f9e484" Sep 09 20:59:13.449678 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.449658 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5tzzp"] Sep 09 20:59:13.456465 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.456442 2538 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5tzzp"] Sep 09 20:59:13.509492 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.509471 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/776b6475-3903-46b3-b7dc-e7987ea5dc89-data\") pod \"postgres-69fb4594cf-9dnnf\" (UID: \"776b6475-3903-46b3-b7dc-e7987ea5dc89\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.509586 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.509525 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d2w7\" (UniqueName: \"kubernetes.io/projected/776b6475-3903-46b3-b7dc-e7987ea5dc89-kube-api-access-4d2w7\") pod \"postgres-69fb4594cf-9dnnf\" (UID: \"776b6475-3903-46b3-b7dc-e7987ea5dc89\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.509877 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.509862 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/776b6475-3903-46b3-b7dc-e7987ea5dc89-data\") pod \"postgres-69fb4594cf-9dnnf\" (UID: \"776b6475-3903-46b3-b7dc-e7987ea5dc89\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.521357 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.521335 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d2w7\" (UniqueName: \"kubernetes.io/projected/776b6475-3903-46b3-b7dc-e7987ea5dc89-kube-api-access-4d2w7\") pod \"postgres-69fb4594cf-9dnnf\" (UID: \"776b6475-3903-46b3-b7dc-e7987ea5dc89\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.662564 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.662498 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:13.937271 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:13.937185 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf"] Sep 09 20:59:13.940644 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:59:13.940611 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776b6475_3903_46b3_b7dc_e7987ea5dc89.slice/crio-84476759ab4294cc9a57ff62c947d2a0fea549ee1b97655a6a347ade2cc2bf19 WatchSource:0}: Error finding container 84476759ab4294cc9a57ff62c947d2a0fea549ee1b97655a6a347ade2cc2bf19: Status 404 returned error can't find the container with id 84476759ab4294cc9a57ff62c947d2a0fea549ee1b97655a6a347ade2cc2bf19 Sep 09 20:59:14.431090 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:14.431052 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" event={"ID":"776b6475-3903-46b3-b7dc-e7987ea5dc89","Type":"ContainerStarted","Data":"84476759ab4294cc9a57ff62c947d2a0fea549ee1b97655a6a347ade2cc2bf19"} Sep 09 20:59:14.874259 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:14.874202 2538 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be4f3ca2-8263-4bf7-8186-73698c800a3f" path="/var/lib/kubelet/pods/be4f3ca2-8263-4bf7-8186-73698c800a3f/volumes" Sep 09 20:59:19.458380 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:19.458343 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" event={"ID":"776b6475-3903-46b3-b7dc-e7987ea5dc89","Type":"ContainerStarted","Data":"0ab2ed46fd047c4e235f3edf0544e45700a53363c5d79b029128d9d0305fd833"} Sep 09 20:59:19.458925 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:19.458455 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:25.493195 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:25.493162 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" Sep 09 20:59:25.521217 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:25.521170 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dnnf" podStartSLOduration=7.609891448 podStartE2EDuration="12.521155011s" totalImagesPullingTime="4.911263563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:19.493007436 +0000 UTC m=+701.111667668" watchObservedRunningTime="2026-09-09 20:59:25.521155011 +0000 UTC m=+707.139815243" Sep 09 20:59:25.924861 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:25.924808 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:26.843391 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:26.843358 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:33.271174 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.271080 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:33.303596 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.303565 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-769dc5bdf-5727q"] Sep 09 20:59:33.306465 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.306449 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.312918 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.312897 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 09 20:59:33.313174 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.313161 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 09 20:59:33.314191 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.314176 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lk5g4\"" Sep 09 20:59:33.314270 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.314197 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 09 20:59:33.329075 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.329052 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769dc5bdf-5727q"] Sep 09 20:59:33.371676 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.371653 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:33.473633 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.473604 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:33.485237 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.485200 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-metrics-certs\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.485342 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.485246 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l56gd\" (UniqueName: \"kubernetes.io/projected/5926e4f9-ebba-44bf-8599-85568b2bd298-kube-api-access-l56gd\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.485342 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.485288 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-webhook-cert\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.500921 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.500883 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77687bdfb8-tpb5r"] Sep 09 20:59:33.504075 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.504060 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.516742 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.516719 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77687bdfb8-tpb5r"] Sep 09 20:59:33.586502 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.586474 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-metrics-certs\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.586646 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.586510 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l56gd\" (UniqueName: \"kubernetes.io/projected/5926e4f9-ebba-44bf-8599-85568b2bd298-kube-api-access-l56gd\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.586646 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.586537 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-webhook-cert\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.589144 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.589115 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-metrics-certs\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.589280 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.589164 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-webhook-cert\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.597338 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.597317 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l56gd\" (UniqueName: \"kubernetes.io/projected/5926e4f9-ebba-44bf-8599-85568b2bd298-kube-api-access-l56gd\") pod \"maas-controller-769dc5bdf-5727q\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.606741 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.606714 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:33.615575 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.615555 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:33.687071 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.687045 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swl5n\" (UniqueName: \"kubernetes.io/projected/90c3c3c7-068d-45e7-81a6-d00455f8c365-kube-api-access-swl5n\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.687179 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.687088 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90c3c3c7-068d-45e7-81a6-d00455f8c365-webhook-cert\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.687179 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.687169 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90c3c3c7-068d-45e7-81a6-d00455f8c365-metrics-certs\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.788176 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.788150 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90c3c3c7-068d-45e7-81a6-d00455f8c365-metrics-certs\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.788329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.788186 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swl5n\" (UniqueName: \"kubernetes.io/projected/90c3c3c7-068d-45e7-81a6-d00455f8c365-kube-api-access-swl5n\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.788329 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.788212 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90c3c3c7-068d-45e7-81a6-d00455f8c365-webhook-cert\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.790797 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.790772 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90c3c3c7-068d-45e7-81a6-d00455f8c365-webhook-cert\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.790882 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.790772 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90c3c3c7-068d-45e7-81a6-d00455f8c365-metrics-certs\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.800334 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.800307 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swl5n\" (UniqueName: \"kubernetes.io/projected/90c3c3c7-068d-45e7-81a6-d00455f8c365-kube-api-access-swl5n\") pod \"maas-controller-77687bdfb8-tpb5r\" (UID: \"90c3c3c7-068d-45e7-81a6-d00455f8c365\") " pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.814008 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.813977 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:33.898634 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:33.898603 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769dc5bdf-5727q"] Sep 09 20:59:33.902342 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:59:33.902317 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5926e4f9_ebba_44bf_8599_85568b2bd298.slice/crio-5eb66f1bae8b6ac467033bef7b37a46dc373ad9a22d26e2c295a49b472c217ef WatchSource:0}: Error finding container 5eb66f1bae8b6ac467033bef7b37a46dc373ad9a22d26e2c295a49b472c217ef: Status 404 returned error can't find the container with id 5eb66f1bae8b6ac467033bef7b37a46dc373ad9a22d26e2c295a49b472c217ef Sep 09 20:59:34.086251 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:34.086202 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77687bdfb8-tpb5r"] Sep 09 20:59:34.089050 ip-10-0-143-6 kubenswrapper[2538]: W0909 20:59:34.089024 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c3c3c7_068d_45e7_81a6_d00455f8c365.slice/crio-33baa0d7ef6ba0cc19071e31fe173aad06e79177edd4813441cb720f34465e6a WatchSource:0}: Error finding container 33baa0d7ef6ba0cc19071e31fe173aad06e79177edd4813441cb720f34465e6a: Status 404 returned error can't find the container with id 33baa0d7ef6ba0cc19071e31fe173aad06e79177edd4813441cb720f34465e6a Sep 09 20:59:34.531561 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:34.531518 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769dc5bdf-5727q" event={"ID":"5926e4f9-ebba-44bf-8599-85568b2bd298","Type":"ContainerStarted","Data":"5eb66f1bae8b6ac467033bef7b37a46dc373ad9a22d26e2c295a49b472c217ef"} Sep 09 20:59:34.532688 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:34.532652 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77687bdfb8-tpb5r" event={"ID":"90c3c3c7-068d-45e7-81a6-d00455f8c365","Type":"ContainerStarted","Data":"33baa0d7ef6ba0cc19071e31fe173aad06e79177edd4813441cb720f34465e6a"} Sep 09 20:59:36.833041 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:36.833002 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:36.947128 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:36.947043 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 20:59:37.552488 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:37.552445 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769dc5bdf-5727q" event={"ID":"5926e4f9-ebba-44bf-8599-85568b2bd298","Type":"ContainerStarted","Data":"2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c"} Sep 09 20:59:37.552764 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:37.552745 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:37.555020 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:37.554990 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77687bdfb8-tpb5r" event={"ID":"90c3c3c7-068d-45e7-81a6-d00455f8c365","Type":"ContainerStarted","Data":"1b5e8a5bf25b0a445d28ef9dcb781726627ccb2a06e5063eccecb11d0514cec4"} Sep 09 20:59:37.555155 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:37.555082 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:37.636841 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:37.636783 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77687bdfb8-tpb5r" podStartSLOduration=2.115553983 podStartE2EDuration="4.636766174s" totalImagesPullingTime="2.521212191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:37.633277614 +0000 UTC m=+719.251937848" watchObservedRunningTime="2026-09-09 20:59:37.636766174 +0000 UTC m=+719.255426406" Sep 09 20:59:37.637254 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:37.637212 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-769dc5bdf-5727q" podStartSLOduration=1.936346045 podStartE2EDuration="4.637204881s" totalImagesPullingTime="2.700858836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:37.582747874 +0000 UTC m=+719.201408107" watchObservedRunningTime="2026-09-09 20:59:37.637204881 +0000 UTC m=+719.255865158" Sep 09 20:59:48.562345 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.562315 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:48.562767 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.562389 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77687bdfb8-tpb5r" Sep 09 20:59:48.643539 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.643460 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769dc5bdf-5727q"] Sep 09 20:59:48.644472 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.644423 2538 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-769dc5bdf-5727q" podUID="5926e4f9-ebba-44bf-8599-85568b2bd298" containerName="manager" containerID="cri-o://2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c" gracePeriod=30 Sep 09 20:59:48.890993 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.890965 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:48.914500 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.914475 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-metrics-certs\") pod \"5926e4f9-ebba-44bf-8599-85568b2bd298\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " Sep 09 20:59:48.914641 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.914558 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l56gd\" (UniqueName: \"kubernetes.io/projected/5926e4f9-ebba-44bf-8599-85568b2bd298-kube-api-access-l56gd\") pod \"5926e4f9-ebba-44bf-8599-85568b2bd298\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " Sep 09 20:59:48.914641 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.914604 2538 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-webhook-cert\") pod \"5926e4f9-ebba-44bf-8599-85568b2bd298\" (UID: \"5926e4f9-ebba-44bf-8599-85568b2bd298\") " Sep 09 20:59:48.918372 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.918321 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5926e4f9-ebba-44bf-8599-85568b2bd298" (UID: "5926e4f9-ebba-44bf-8599-85568b2bd298"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:59:48.918503 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.918361 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5926e4f9-ebba-44bf-8599-85568b2bd298" (UID: "5926e4f9-ebba-44bf-8599-85568b2bd298"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:59:48.918575 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:48.918503 2538 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5926e4f9-ebba-44bf-8599-85568b2bd298-kube-api-access-l56gd" (OuterVolumeSpecName: "kube-api-access-l56gd") pod "5926e4f9-ebba-44bf-8599-85568b2bd298" (UID: "5926e4f9-ebba-44bf-8599-85568b2bd298"). InnerVolumeSpecName "kube-api-access-l56gd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:59:49.015804 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.015777 2538 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l56gd\" (UniqueName: \"kubernetes.io/projected/5926e4f9-ebba-44bf-8599-85568b2bd298-kube-api-access-l56gd\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:59:49.015804 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.015800 2538 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-webhook-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:59:49.015804 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.015809 2538 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5926e4f9-ebba-44bf-8599-85568b2bd298-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Sep 09 20:59:49.607541 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.607511 2538 generic.go:356] "Generic (PLEG): container finished" podID="5926e4f9-ebba-44bf-8599-85568b2bd298" containerID="2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c" exitCode=0 Sep 09 20:59:49.607930 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.607556 2538 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769dc5bdf-5727q" Sep 09 20:59:49.607930 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.607602 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769dc5bdf-5727q" event={"ID":"5926e4f9-ebba-44bf-8599-85568b2bd298","Type":"ContainerDied","Data":"2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c"} Sep 09 20:59:49.607930 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.607652 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769dc5bdf-5727q" event={"ID":"5926e4f9-ebba-44bf-8599-85568b2bd298","Type":"ContainerDied","Data":"5eb66f1bae8b6ac467033bef7b37a46dc373ad9a22d26e2c295a49b472c217ef"} Sep 09 20:59:49.607930 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.607673 2538 scope.go:117] "RemoveContainer" containerID="2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c" Sep 09 20:59:49.616553 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.616539 2538 scope.go:117] "RemoveContainer" containerID="2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c" Sep 09 20:59:49.616811 ip-10-0-143-6 kubenswrapper[2538]: E0909 20:59:49.616796 2538 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c\": container with ID starting with 2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c not found: ID does not exist" containerID="2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c" Sep 09 20:59:49.616869 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.616821 2538 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c"} err="failed to get container status \"2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c\": rpc error: code = NotFound desc = could not find container \"2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c\": container with ID starting with 2f099a2c29795fb3f84d202d4540b94486b080b1a5020493563df1e7d632282c not found: ID does not exist" Sep 09 20:59:49.632595 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.632573 2538 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769dc5bdf-5727q"] Sep 09 20:59:49.637117 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:49.637096 2538 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-769dc5bdf-5727q"] Sep 09 20:59:50.870004 ip-10-0-143-6 kubenswrapper[2538]: I0909 20:59:50.869967 2538 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5926e4f9-ebba-44bf-8599-85568b2bd298" path="/var/lib/kubelet/pods/5926e4f9-ebba-44bf-8599-85568b2bd298/volumes" Sep 09 21:00:08.813571 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:08.813539 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:08.944973 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:08.944937 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:10.252112 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.252066 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:10.323703 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.323670 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb"] Sep 09 21:00:10.324075 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.324059 2538 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5926e4f9-ebba-44bf-8599-85568b2bd298" containerName="manager" Sep 09 21:00:10.324122 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.324079 2538 state_mem.go:107] "Deleted CPUSet assignment" podUID="5926e4f9-ebba-44bf-8599-85568b2bd298" containerName="manager" Sep 09 21:00:10.324155 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.324145 2538 memory_manager.go:368] "RemoveStaleState removing state" podUID="5926e4f9-ebba-44bf-8599-85568b2bd298" containerName="manager" Sep 09 21:00:10.329897 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.329872 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.330530 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.330497 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:10.332398 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.332380 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 09 21:00:10.332798 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.332780 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 09 21:00:10.338537 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.338505 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb"] Sep 09 21:00:10.402876 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.402850 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd63f92e-d534-4a5b-9841-e563b670e1ae-openshift-service-ca\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.403039 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.402898 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-metrics-certs\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.403039 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.402919 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-maas-api-tls\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.403039 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.403024 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2zw\" (UniqueName: \"kubernetes.io/projected/cd63f92e-d534-4a5b-9841-e563b670e1ae-kube-api-access-6g2zw\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.503974 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.503906 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-maas-api-tls\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.503974 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.503952 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2zw\" (UniqueName: \"kubernetes.io/projected/cd63f92e-d534-4a5b-9841-e563b670e1ae-kube-api-access-6g2zw\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.504179 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.503992 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd63f92e-d534-4a5b-9841-e563b670e1ae-openshift-service-ca\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.504179 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.504028 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-metrics-certs\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.504179 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:10.504047 2538 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Sep 09 21:00:10.504179 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:10.504116 2538 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-maas-api-tls podName:cd63f92e-d534-4a5b-9841-e563b670e1ae nodeName:}" failed. No retries permitted until 2026-09-09 21:00:11.004094925 +0000 UTC m=+752.622755145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-maas-api-tls") pod "maas-api-5b8dcd48fb-jg7mb" (UID: "cd63f92e-d534-4a5b-9841-e563b670e1ae") : secret "maas-api-serving-cert" not found Sep 09 21:00:10.504694 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.504673 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd63f92e-d534-4a5b-9841-e563b670e1ae-openshift-service-ca\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.506335 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.506319 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-metrics-certs\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.514008 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.513985 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2zw\" (UniqueName: \"kubernetes.io/projected/cd63f92e-d534-4a5b-9841-e563b670e1ae-kube-api-access-6g2zw\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:10.596942 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.596909 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:10.692313 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.692285 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:10.822534 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.822503 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:10.920032 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:10.919998 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:11.007528 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.007498 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-maas-api-tls\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:11.010003 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.009979 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd63f92e-d534-4a5b-9841-e563b670e1ae-maas-api-tls\") pod \"maas-api-5b8dcd48fb-jg7mb\" (UID: \"cd63f92e-d534-4a5b-9841-e563b670e1ae\") " pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:11.051319 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.051246 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5"] Sep 09 21:00:11.054576 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.054560 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.059889 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.059872 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 09 21:00:11.060161 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.060146 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4gq6k\"" Sep 09 21:00:11.151874 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.151807 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5"] Sep 09 21:00:11.209129 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.209093 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e44051-9885-4209-b5e2-c43d9598f9e7-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-95gb5\" (UID: \"56e44051-9885-4209-b5e2-c43d9598f9e7\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.209129 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.209130 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mpv2\" (UniqueName: \"kubernetes.io/projected/56e44051-9885-4209-b5e2-c43d9598f9e7-kube-api-access-4mpv2\") pod \"payload-pre-processing-59fbc47bf8-95gb5\" (UID: \"56e44051-9885-4209-b5e2-c43d9598f9e7\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.242280 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.242214 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:11.310362 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.310338 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e44051-9885-4209-b5e2-c43d9598f9e7-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-95gb5\" (UID: \"56e44051-9885-4209-b5e2-c43d9598f9e7\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.310772 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.310376 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mpv2\" (UniqueName: \"kubernetes.io/projected/56e44051-9885-4209-b5e2-c43d9598f9e7-kube-api-access-4mpv2\") pod \"payload-pre-processing-59fbc47bf8-95gb5\" (UID: \"56e44051-9885-4209-b5e2-c43d9598f9e7\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.311495 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.311473 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e44051-9885-4209-b5e2-c43d9598f9e7-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-95gb5\" (UID: \"56e44051-9885-4209-b5e2-c43d9598f9e7\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.324214 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.324176 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mpv2\" (UniqueName: \"kubernetes.io/projected/56e44051-9885-4209-b5e2-c43d9598f9e7-kube-api-access-4mpv2\") pod \"payload-pre-processing-59fbc47bf8-95gb5\" (UID: \"56e44051-9885-4209-b5e2-c43d9598f9e7\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.365340 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.365317 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:11.530486 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.530391 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb"] Sep 09 21:00:11.532805 ip-10-0-143-6 kubenswrapper[2538]: W0909 21:00:11.532766 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd63f92e_d534_4a5b_9841_e563b670e1ae.slice/crio-f804217feeba79a6e2f3505c3028dec348c1190773657b59ea52d94cf6de4738 WatchSource:0}: Error finding container f804217feeba79a6e2f3505c3028dec348c1190773657b59ea52d94cf6de4738: Status 404 returned error can't find the container with id f804217feeba79a6e2f3505c3028dec348c1190773657b59ea52d94cf6de4738 Sep 09 21:00:11.645862 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.639779 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5"] Sep 09 21:00:11.704216 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.704136 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" event={"ID":"cd63f92e-d534-4a5b-9841-e563b670e1ae","Type":"ContainerStarted","Data":"f804217feeba79a6e2f3505c3028dec348c1190773657b59ea52d94cf6de4738"} Sep 09 21:00:11.705517 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:11.705491 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" event={"ID":"56e44051-9885-4209-b5e2-c43d9598f9e7","Type":"ContainerStarted","Data":"56ced90f1e504c7e7cfdaeefa1eebe92865b382097298c2bfc24b26f2ee2a3c3"} Sep 09 21:00:15.729547 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:15.729509 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" event={"ID":"cd63f92e-d534-4a5b-9841-e563b670e1ae","Type":"ContainerStarted","Data":"8259d0ba04c5bef35622de13ce6b25a1bf573d206d8ff8c0e8972a765ec60b6c"} Sep 09 21:00:15.730007 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:15.729626 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:15.731239 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:15.731206 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" event={"ID":"56e44051-9885-4209-b5e2-c43d9598f9e7","Type":"ContainerStarted","Data":"c6474e2bfd3827fae44093577521445a0abb899f5ae40f306156c8a12437fcb0"} Sep 09 21:00:15.731335 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:15.731294 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:15.748953 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:15.748902 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" podStartSLOduration=2.608599665 podStartE2EDuration="5.748873342s" totalImagesPullingTime="3.140273677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:15.747668965 +0000 UTC m=+757.366329199" watchObservedRunningTime="2026-09-09 21:00:15.748873342 +0000 UTC m=+757.367533580" Sep 09 21:00:15.769959 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:15.769917 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" podStartSLOduration=1.742947413 podStartE2EDuration="4.769903348s" totalImagesPullingTime="3.026955935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:15.768811952 +0000 UTC m=+757.387472184" watchObservedRunningTime="2026-09-09 21:00:15.769903348 +0000 UTC m=+757.388563557" Sep 09 21:00:21.741729 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:21.741698 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b8dcd48fb-jg7mb" Sep 09 21:00:22.718721 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:22.718682 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:22.863726 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:22.863693 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:26.737811 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:26.737779 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-95gb5" Sep 09 21:00:27.631566 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.631534 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p"] Sep 09 21:00:27.636035 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.636018 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.643459 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.643434 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 09 21:00:27.643587 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.643434 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gkdq5\"" Sep 09 21:00:27.643587 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.643439 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 21:00:27.651653 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.651632 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.651751 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.651696 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.651751 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.651732 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2bkb\" (UniqueName: \"kubernetes.io/projected/4ba47bf1-3ed3-4005-a265-7b12d74eab75-kube-api-access-s2bkb\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.651821 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.651752 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.651821 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.651769 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.651821 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.651789 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ba47bf1-3ed3-4005-a265-7b12d74eab75-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.655406 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.655392 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 21:00:27.667497 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.667470 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p"] Sep 09 21:00:27.694378 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.694353 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:27.752623 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.752598 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ba47bf1-3ed3-4005-a265-7b12d74eab75-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.752969 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.752654 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.752969 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.752697 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.752969 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.752715 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2bkb\" (UniqueName: \"kubernetes.io/projected/4ba47bf1-3ed3-4005-a265-7b12d74eab75-kube-api-access-s2bkb\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.752969 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.752732 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.752969 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.752749 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.753250 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.753153 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.753320 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.753251 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.753320 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.753274 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.755058 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.755038 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ba47bf1-3ed3-4005-a265-7b12d74eab75-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.755356 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.755338 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ba47bf1-3ed3-4005-a265-7b12d74eab75-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.765580 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.765555 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2bkb\" (UniqueName: \"kubernetes.io/projected/4ba47bf1-3ed3-4005-a265-7b12d74eab75-kube-api-access-s2bkb\") pod \"e2e-distinct-2-simulated-kserve-859856b874-p4p9p\" (UID: \"4ba47bf1-3ed3-4005-a265-7b12d74eab75\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:27.946836 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:27.946760 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:28.230193 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:28.230168 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p"] Sep 09 21:00:28.231709 ip-10-0-143-6 kubenswrapper[2538]: W0909 21:00:28.231685 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba47bf1_3ed3_4005_a265_7b12d74eab75.slice/crio-1a4f8a38092d779cf8ef817eb8231b4d76c5421ffbd3fe2d4c0385fff88d2096 WatchSource:0}: Error finding container 1a4f8a38092d779cf8ef817eb8231b4d76c5421ffbd3fe2d4c0385fff88d2096: Status 404 returned error can't find the container with id 1a4f8a38092d779cf8ef817eb8231b4d76c5421ffbd3fe2d4c0385fff88d2096 Sep 09 21:00:28.791471 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:28.791428 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerStarted","Data":"1a4f8a38092d779cf8ef817eb8231b4d76c5421ffbd3fe2d4c0385fff88d2096"} Sep 09 21:00:29.797607 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:29.797516 2538 generic.go:356] "Generic (PLEG): container finished" podID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" containerID="8ddc0425fbea5a07b12691af185c6087ddc406bcac53ba9efd4f2b3734d7964f" exitCode=0 Sep 09 21:00:29.797607 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:29.797562 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerDied","Data":"8ddc0425fbea5a07b12691af185c6087ddc406bcac53ba9efd4f2b3734d7964f"} Sep 09 21:00:29.797993 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:29.797838 2538 scope.go:117] "RemoveContainer" containerID="8ddc0425fbea5a07b12691af185c6087ddc406bcac53ba9efd4f2b3734d7964f" Sep 09 21:00:30.804203 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:30.804168 2538 generic.go:356] "Generic (PLEG): container finished" podID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" containerID="c4fa8927b1d14d1c741bb751c16955a499d07508af3e0cea16c59fc34d2aa33a" exitCode=0 Sep 09 21:00:30.804641 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:30.804211 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerDied","Data":"c4fa8927b1d14d1c741bb751c16955a499d07508af3e0cea16c59fc34d2aa33a"} Sep 09 21:00:30.804641 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:30.804307 2538 scope.go:117] "RemoveContainer" containerID="8ddc0425fbea5a07b12691af185c6087ddc406bcac53ba9efd4f2b3734d7964f" Sep 09 21:00:30.804641 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:30.804498 2538 scope.go:117] "RemoveContainer" containerID="c4fa8927b1d14d1c741bb751c16955a499d07508af3e0cea16c59fc34d2aa33a" Sep 09 21:00:30.804780 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:30.804711 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:00:31.810734 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:31.810712 2538 scope.go:117] "RemoveContainer" containerID="c4fa8927b1d14d1c741bb751c16955a499d07508af3e0cea16c59fc34d2aa33a" Sep 09 21:00:31.811163 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:31.810875 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:00:32.807388 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:32.807358 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:37.947963 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:37.947931 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:37.948399 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:37.947987 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:37.948399 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:37.948004 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:37.948399 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:37.948017 2538 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:37.948399 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:37.948380 2538 scope.go:117] "RemoveContainer" containerID="c4fa8927b1d14d1c741bb751c16955a499d07508af3e0cea16c59fc34d2aa33a" Sep 09 21:00:37.948569 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:37.948552 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:00:49.944782 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:49.944742 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:00:50.865582 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:50.865548 2538 scope.go:117] "RemoveContainer" containerID="c4fa8927b1d14d1c741bb751c16955a499d07508af3e0cea16c59fc34d2aa33a" Sep 09 21:00:51.899157 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:51.899125 2538 generic.go:356] "Generic (PLEG): container finished" podID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" containerID="fa303633d7f02239295e7da80dd9b8cb874842ceccce013ef5a33a60d081f8fd" exitCode=0 Sep 09 21:00:51.899611 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:51.899199 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerDied","Data":"fa303633d7f02239295e7da80dd9b8cb874842ceccce013ef5a33a60d081f8fd"} Sep 09 21:00:51.899611 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:51.899270 2538 scope.go:117] "RemoveContainer" containerID="c4fa8927b1d14d1c741bb751c16955a499d07508af3e0cea16c59fc34d2aa33a" Sep 09 21:00:51.899611 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:51.899557 2538 scope.go:117] "RemoveContainer" containerID="fa303633d7f02239295e7da80dd9b8cb874842ceccce013ef5a33a60d081f8fd" Sep 09 21:00:51.900061 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:51.899737 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:00:57.947781 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:57.947746 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:57.948359 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:57.947869 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:57.948359 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:57.947904 2538 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:57.948359 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:57.948050 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:00:57.948359 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:57.948184 2538 scope.go:117] "RemoveContainer" containerID="fa303633d7f02239295e7da80dd9b8cb874842ceccce013ef5a33a60d081f8fd" Sep 09 21:00:57.948508 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:57.948394 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:00:58.931846 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:00:58.931820 2538 scope.go:117] "RemoveContainer" containerID="fa303633d7f02239295e7da80dd9b8cb874842ceccce013ef5a33a60d081f8fd" Sep 09 21:00:58.932019 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:00:58.932000 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:01:00.746071 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:00.745993 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:01:03.582164 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.582129 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n"] Sep 09 21:01:03.585593 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.585567 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.588570 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.588549 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 09 21:01:03.597077 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.597054 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n"] Sep 09 21:01:03.670445 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.670415 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.670445 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.670454 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw9bm\" (UniqueName: \"kubernetes.io/projected/8674d4b2-3514-4451-8b26-58c39c98c172-kube-api-access-pw9bm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.670631 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.670481 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.670631 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.670510 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.670631 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.670529 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.670631 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.670610 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8674d4b2-3514-4451-8b26-58c39c98c172-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771111 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771086 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771276 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771116 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw9bm\" (UniqueName: \"kubernetes.io/projected/8674d4b2-3514-4451-8b26-58c39c98c172-kube-api-access-pw9bm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771276 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771137 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771276 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771155 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771276 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771171 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771276 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771261 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8674d4b2-3514-4451-8b26-58c39c98c172-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771642 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771610 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771754 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771685 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.771901 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.771848 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.773840 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.773823 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8674d4b2-3514-4451-8b26-58c39c98c172-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.773950 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.773929 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8674d4b2-3514-4451-8b26-58c39c98c172-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.780113 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.780093 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw9bm\" (UniqueName: \"kubernetes.io/projected/8674d4b2-3514-4451-8b26-58c39c98c172-kube-api-access-pw9bm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n\" (UID: \"8674d4b2-3514-4451-8b26-58c39c98c172\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:03.895720 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:03.895658 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:04.010558 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:04.010528 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:01:04.171544 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:04.171521 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n"] Sep 09 21:01:04.173848 ip-10-0-143-6 kubenswrapper[2538]: W0909 21:01:04.173819 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8674d4b2_3514_4451_8b26_58c39c98c172.slice/crio-fb59c0f4cb464856a601838fb144752d736b6d81b2e0471cc8a39c6fe8ac8fb5 WatchSource:0}: Error finding container fb59c0f4cb464856a601838fb144752d736b6d81b2e0471cc8a39c6fe8ac8fb5: Status 404 returned error can't find the container with id fb59c0f4cb464856a601838fb144752d736b6d81b2e0471cc8a39c6fe8ac8fb5 Sep 09 21:01:04.961276 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:04.961240 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" event={"ID":"8674d4b2-3514-4451-8b26-58c39c98c172","Type":"ContainerStarted","Data":"dc49112a926b67d89d1eeae4697a22d36778202d57890d836f68f6d231ee57a2"} Sep 09 21:01:04.961645 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:04.961288 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" event={"ID":"8674d4b2-3514-4451-8b26-58c39c98c172","Type":"ContainerStarted","Data":"fb59c0f4cb464856a601838fb144752d736b6d81b2e0471cc8a39c6fe8ac8fb5"} Sep 09 21:01:05.015583 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:05.015532 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" podStartSLOduration=1.8214683630000001 podStartE2EDuration="2.015516017s" totalImagesPullingTime="194.047654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:05.014189401 +0000 UTC m=+806.632849641" watchObservedRunningTime="2026-09-09 21:01:05.015516017 +0000 UTC m=+806.634176248" Sep 09 21:01:09.039487 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:09.039438 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:01:10.638109 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:10.638060 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:01:13.865650 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:13.865623 2538 scope.go:117] "RemoveContainer" containerID="fa303633d7f02239295e7da80dd9b8cb874842ceccce013ef5a33a60d081f8fd" Sep 09 21:01:13.866025 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:13.866005 2538 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:01:13.896188 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:13.896154 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:13.896365 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:13.896241 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:13.908535 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:13.908512 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:14.018983 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:14.018953 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n" Sep 09 21:01:15.014463 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:15.014424 2538 generic.go:356] "Generic (PLEG): container finished" podID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" containerID="f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6" exitCode=0 Sep 09 21:01:15.014952 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:15.014504 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerDied","Data":"f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6"} Sep 09 21:01:15.014952 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:15.014587 2538 scope.go:117] "RemoveContainer" containerID="fa303633d7f02239295e7da80dd9b8cb874842ceccce013ef5a33a60d081f8fd" Sep 09 21:01:15.015071 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:15.015025 2538 scope.go:117] "RemoveContainer" containerID="f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6" Sep 09 21:01:15.015353 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:01:15.015330 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:01:17.771378 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:17.771345 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:01:17.947828 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:17.947786 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:17.948009 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:17.947864 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:17.948009 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:17.947887 2538 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:17.948009 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:17.947907 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:17.948343 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:17.948323 2538 scope.go:117] "RemoveContainer" containerID="f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6" Sep 09 21:01:17.948556 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:01:17.948532 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:01:24.867423 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:24.867384 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:01:28.866887 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:28.866857 2538 scope.go:117] "RemoveContainer" containerID="f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6" Sep 09 21:01:28.867275 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:01:28.867055 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:01:42.865532 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:42.865505 2538 scope.go:117] "RemoveContainer" containerID="f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6" Sep 09 21:01:42.866051 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:01:42.865744 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:01:54.865526 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:54.865497 2538 scope.go:117] "RemoveContainer" containerID="f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6" Sep 09 21:01:55.205027 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:55.204997 2538 generic.go:356] "Generic (PLEG): container finished" podID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" exitCode=0 Sep 09 21:01:55.205134 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:55.205070 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerDied","Data":"a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55"} Sep 09 21:01:55.205134 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:55.205131 2538 scope.go:117] "RemoveContainer" containerID="f151a9588495d5219d7104352b48f32e750a7a44adb0cbc531556eec1f2ff9c6" Sep 09 21:01:55.205525 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:55.205499 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:01:55.205742 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:01:55.205718 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:01:57.947098 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:57.947061 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:57.947575 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:57.947118 2538 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:57.947575 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:57.947134 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:57.947575 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:57.947150 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:01:57.947575 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:01:57.947475 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:01:57.947772 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:01:57.947642 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:02:08.540952 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:08.540914 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:10.357876 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:10.357835 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:10.864810 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:10.864782 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:02:10.864970 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:02:10.864953 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:02:14.593415 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:14.593378 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:14.671408 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:14.671381 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:02:24.864886 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:24.864855 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:02:24.865300 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:02:24.865075 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:02:35.864731 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:35.864641 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:02:35.865152 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:02:35.864877 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:02:50.865132 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:02:50.865097 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:02:50.865606 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:02:50.865368 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:03:05.864524 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:05.864495 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:03:05.864912 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:03:05.864661 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:03:20.865197 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:20.865167 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:03:21.598726 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:21.598694 2538 generic.go:356] "Generic (PLEG): container finished" podID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" exitCode=0 Sep 09 21:03:21.598889 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:21.598757 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerDied","Data":"22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041"} Sep 09 21:03:21.598889 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:21.598825 2538 scope.go:117] "RemoveContainer" containerID="a9742e7eb497c4107fd00e375baaf0ab8838ce88a9acaf1399aac6e8c39aeb55" Sep 09 21:03:21.599170 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:21.599149 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:03:21.599399 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:03:21.599377 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:03:27.947615 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:27.947580 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:03:27.948096 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:27.947648 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:03:27.948096 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:27.947672 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:03:27.948096 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:27.947692 2538 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:03:27.948096 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:27.948003 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:03:27.948266 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:03:27.948165 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:03:40.864701 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:40.864668 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:03:40.865062 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:03:40.864867 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:03:53.865090 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:03:53.865054 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:03:53.865496 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:03:53.865295 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:04:06.519533 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:06.519448 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:06.603371 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:06.603341 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:07.864994 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:07.864966 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:04:07.865396 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:04:07.865138 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:04:15.651915 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:15.651881 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:18.866763 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:18.866735 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:04:18.867152 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:04:18.866932 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:04:33.864559 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:33.864527 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:04:33.864971 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:04:33.864696 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:04:46.865461 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:46.865429 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:04:46.865908 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:04:46.865609 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:04:48.983570 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:48.983534 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:04:49.065329 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:04:49.065294 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:00.865387 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:00.865357 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:05:00.865762 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:05:00.865532 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:05:13.824690 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:13.824654 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:14.864799 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:14.864764 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:05:14.865169 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:05:14.864953 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:05:27.865080 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:27.865050 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:05:27.865545 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:05:27.865252 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:05:33.872514 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:33.872443 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tj8pt_cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5/manager/0.log" Sep 09 21:05:33.988257 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:33.988212 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-krvtr_bbdc23d5-8f42-4b6b-9ede-412e927b145d/manager/0.log" Sep 09 21:05:34.102568 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:34.102541 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77687bdfb8-tpb5r_90c3c3c7-068d-45e7-81a6-d00455f8c365/manager/0.log" Sep 09 21:05:34.460808 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:34.460779 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97cf5684c-5sfkf_48345d2d-d88c-4551-8cf5-c4d97de63c4f/copy-manifests/0.log" Sep 09 21:05:34.466790 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:34.466768 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97cf5684c-5sfkf_48345d2d-d88c-4551-8cf5-c4d97de63c4f/manager/0.log" Sep 09 21:05:35.067283 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:35.067252 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5b8dcd48fb-jg7mb_cd63f92e-d534-4a5b-9841-e563b670e1ae/maas-api/0.log" Sep 09 21:05:35.176213 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:35.176170 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-9dnnf_776b6475-3903-46b3-b7dc-e7987ea5dc89/postgres/0.log" Sep 09 21:05:36.248505 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:36.248475 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-274nd_53095058-cb0c-4715-a856-020743bb74de/manager/0.log" Sep 09 21:05:36.496241 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:36.496197 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-bpm7r_7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1/manager/0.log" Sep 09 21:05:37.673627 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:37.673596 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-95gb5_56e44051-9885-4209-b5e2-c43d9598f9e7/payload-pre-processing/0.log" Sep 09 21:05:37.905277 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:37.905249 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d949964-m9n92_e4728dbe-431a-44c6-93e1-889c4e86b95e/router/0.log" Sep 09 21:05:38.255197 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:38.255171 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-859856b874-p4p9p_4ba47bf1-3ed3-4005-a265-7b12d74eab75/main/5.log" Sep 09 21:05:38.711576 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:38.711540 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhwg7n_8674d4b2-3514-4451-8b26-58c39c98c172/main/0.log" Sep 09 21:05:42.865454 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:42.865417 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:05:42.865931 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:05:42.865659 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:05:44.786448 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:44.786406 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:44.877787 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:44.877761 2538 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" limit=1024 Sep 09 21:05:47.880506 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:47.880477 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fd8jq_291d4321-4e22-4921-a84c-707139c9b66a/global-pull-secret-syncer/0.log" Sep 09 21:05:48.024282 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:48.024252 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dcdj7_721b84f2-9419-4d85-bc3f-d3ed7f94d9aa/konnectivity-agent/0.log" Sep 09 21:05:48.166128 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:48.166052 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-6.ec2.internal_80bed961383d1f2e370365643dbe1e93/haproxy/0.log" Sep 09 21:05:53.193391 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:53.193367 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-274nd_53095058-cb0c-4715-a856-020743bb74de/manager/0.log" Sep 09 21:05:53.266897 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:53.266872 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-bpm7r_7cefd40f-ce8f-4d50-9be1-dc1ff25ae0a1/manager/0.log" Sep 09 21:05:53.865333 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:53.865300 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:05:53.865538 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:05:53.865520 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:05:54.907399 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:54.907369 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9csq9_887c8b15-bdcd-417c-bb15-d24a989799ea/kube-state-metrics/0.log" Sep 09 21:05:54.938419 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:54.938394 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9csq9_887c8b15-bdcd-417c-bb15-d24a989799ea/kube-rbac-proxy-main/0.log" Sep 09 21:05:54.967022 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:54.966995 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-9csq9_887c8b15-bdcd-417c-bb15-d24a989799ea/kube-rbac-proxy-self/0.log" Sep 09 21:05:55.003026 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.003002 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6bf66bb88d-n2nf7_f21313c6-3913-4f62-9e0c-dbac19ffbb94/metrics-server/0.log" Sep 09 21:05:55.088790 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.088763 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fznkn_1aeaf8df-597d-405c-b0ff-ea832b3cdefe/node-exporter/0.log" Sep 09 21:05:55.115498 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.115474 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fznkn_1aeaf8df-597d-405c-b0ff-ea832b3cdefe/kube-rbac-proxy/0.log" Sep 09 21:05:55.137810 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.137788 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fznkn_1aeaf8df-597d-405c-b0ff-ea832b3cdefe/init-textfile/0.log" Sep 09 21:05:55.431606 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.431576 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7c6b38-04ec-4a48-b7e9-95e67a5b340e/prometheus/0.log" Sep 09 21:05:55.456274 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.456253 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7c6b38-04ec-4a48-b7e9-95e67a5b340e/config-reloader/0.log" Sep 09 21:05:55.486747 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.486721 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7c6b38-04ec-4a48-b7e9-95e67a5b340e/thanos-sidecar/0.log" Sep 09 21:05:55.568902 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.568876 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7c6b38-04ec-4a48-b7e9-95e67a5b340e/kube-rbac-proxy-web/0.log" Sep 09 21:05:55.634216 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.634185 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7c6b38-04ec-4a48-b7e9-95e67a5b340e/kube-rbac-proxy/0.log" Sep 09 21:05:55.696763 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.696692 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7c6b38-04ec-4a48-b7e9-95e67a5b340e/kube-rbac-proxy-thanos/0.log" Sep 09 21:05:55.721131 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:55.721101 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7c6b38-04ec-4a48-b7e9-95e67a5b340e/init-config-reloader/0.log" Sep 09 21:05:57.532644 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.532611 2538 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj"] Sep 09 21:05:57.536041 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.536025 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.539125 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.539107 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh5hz\"/\"openshift-service-ca.crt\"" Sep 09 21:05:57.540248 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.540214 2538 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh5hz\"/\"kube-root-ca.crt\"" Sep 09 21:05:57.540329 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.540214 2538 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qh5hz\"/\"default-dockercfg-k6l4r\"" Sep 09 21:05:57.549768 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.549749 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj"] Sep 09 21:05:57.625520 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.625493 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-podres\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.625667 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.625530 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-proc\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.625667 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.625556 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xltp\" (UniqueName: \"kubernetes.io/projected/d9a2b710-7d65-42a7-91d9-31049681d92a-kube-api-access-5xltp\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.625667 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.625574 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-sys\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.625667 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.625619 2538 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-lib-modules\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.726912 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.726885 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-podres\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727040 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.726921 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-proc\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727040 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.726946 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xltp\" (UniqueName: \"kubernetes.io/projected/d9a2b710-7d65-42a7-91d9-31049681d92a-kube-api-access-5xltp\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727040 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.726965 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-sys\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727040 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.727020 2538 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-lib-modules\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727040 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.727027 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-proc\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727205 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.727064 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-podres\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727205 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.727125 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-sys\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.727205 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.727138 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9a2b710-7d65-42a7-91d9-31049681d92a-lib-modules\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.736689 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.736658 2538 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xltp\" (UniqueName: \"kubernetes.io/projected/d9a2b710-7d65-42a7-91d9-31049681d92a-kube-api-access-5xltp\") pod \"perf-node-gather-daemonset-ck6rj\" (UID: \"d9a2b710-7d65-42a7-91d9-31049681d92a\") " pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:57.845118 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:57.845094 2538 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:58.115773 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:58.115748 2538 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj"] Sep 09 21:05:58.117919 ip-10-0-143-6 kubenswrapper[2538]: W0909 21:05:58.117882 2538 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd9a2b710_7d65_42a7_91d9_31049681d92a.slice/crio-130dec2d26d69912a2e63f956d3ab17c7f5bcbf4d45376f6879da533ef6843bc WatchSource:0}: Error finding container 130dec2d26d69912a2e63f956d3ab17c7f5bcbf4d45376f6879da533ef6843bc: Status 404 returned error can't find the container with id 130dec2d26d69912a2e63f956d3ab17c7f5bcbf4d45376f6879da533ef6843bc Sep 09 21:05:58.321442 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:58.321406 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" event={"ID":"d9a2b710-7d65-42a7-91d9-31049681d92a","Type":"ContainerStarted","Data":"ff5630816cc9c64984801ab58deafbea783f03e549bf2399b77224720ef6ed09"} Sep 09 21:05:58.321618 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:58.321477 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" event={"ID":"d9a2b710-7d65-42a7-91d9-31049681d92a","Type":"ContainerStarted","Data":"130dec2d26d69912a2e63f956d3ab17c7f5bcbf4d45376f6879da533ef6843bc"} Sep 09 21:05:58.321618 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:58.321536 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:05:58.342653 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:05:58.342608 2538 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" podStartSLOduration=1.342594604 podStartE2EDuration="1.342594604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:05:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:05:58.341467371 +0000 UTC m=+1099.960127604" watchObservedRunningTime="2026-09-09 21:05:58.342594604 +0000 UTC m=+1099.961254836" Sep 09 21:06:00.174276 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:00.174244 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m2r86_88c681c3-1804-4094-bb89-687d2fdb9b8a/dns/0.log" Sep 09 21:06:00.197057 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:00.197029 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m2r86_88c681c3-1804-4094-bb89-687d2fdb9b8a/kube-rbac-proxy/0.log" Sep 09 21:06:00.318178 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:00.318151 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wjgh2_03b239fd-3642-4f5d-9380-8c7823fd82ed/dns-node-resolver/0.log" Sep 09 21:06:00.846739 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:00.846704 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d8577cd6c-8wblz_9ac22653-b227-4818-99c2-a493ca64791c/registry/0.log" Sep 09 21:06:00.924356 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:00.924328 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vgpth_f0933970-91fd-4048-b522-eaba643670bb/node-ca/0.log" Sep 09 21:06:02.027882 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:02.027853 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-95gb5_56e44051-9885-4209-b5e2-c43d9598f9e7/payload-pre-processing/0.log" Sep 09 21:06:02.086117 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:02.086092 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d949964-m9n92_e4728dbe-431a-44c6-93e1-889c4e86b95e/router/0.log" Sep 09 21:06:02.768086 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:02.768056 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-49xnp_ef7c014b-1fab-4d97-8831-095a7712283b/serve-healthcheck-canary/0.log" Sep 09 21:06:03.502678 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:03.502640 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xqx4m_171176d5-bac7-41bf-921d-b5e44858e61f/kube-rbac-proxy/0.log" Sep 09 21:06:03.525408 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:03.525383 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xqx4m_171176d5-bac7-41bf-921d-b5e44858e61f/exporter/0.log" Sep 09 21:06:03.547724 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:03.547702 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xqx4m_171176d5-bac7-41bf-921d-b5e44858e61f/extractor/0.log" Sep 09 21:06:04.334998 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:04.334975 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qh5hz/perf-node-gather-daemonset-ck6rj" Sep 09 21:06:05.713370 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:05.713338 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tj8pt_cd4c6ee2-f597-4a8f-a5c9-32a073e2bbb5/manager/0.log" Sep 09 21:06:05.749823 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:05.749791 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-krvtr_bbdc23d5-8f42-4b6b-9ede-412e927b145d/manager/0.log" Sep 09 21:06:05.780310 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:05.780203 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77687bdfb8-tpb5r_90c3c3c7-068d-45e7-81a6-d00455f8c365/manager/0.log" Sep 09 21:06:05.889891 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:05.889864 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97cf5684c-5sfkf_48345d2d-d88c-4551-8cf5-c4d97de63c4f/manager/0.log" Sep 09 21:06:05.910273 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:05.910249 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97cf5684c-5sfkf_48345d2d-d88c-4551-8cf5-c4d97de63c4f/copy-manifests/0.log" Sep 09 21:06:06.865643 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:06.865616 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:06:07.361883 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.361855 2538 generic.go:356] "Generic (PLEG): container finished" podID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" containerID="54e9e6378739517d241a9fb5ac12b91cfe715d8a118736a75c9562ab4651ead7" exitCode=0 Sep 09 21:06:07.362051 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.361933 2538 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" event={"ID":"4ba47bf1-3ed3-4005-a265-7b12d74eab75","Type":"ContainerDied","Data":"54e9e6378739517d241a9fb5ac12b91cfe715d8a118736a75c9562ab4651ead7"} Sep 09 21:06:07.362051 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.362025 2538 scope.go:117] "RemoveContainer" containerID="22b7f1e1bae6617159f78a7e1462911f6f9ad6be12ec41783b207d44b2fe3041" Sep 09 21:06:07.362481 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.362463 2538 scope.go:117] "RemoveContainer" containerID="54e9e6378739517d241a9fb5ac12b91cfe715d8a118736a75c9562ab4651ead7" Sep 09 21:06:07.362684 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:06:07.362658 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:06:07.443143 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.443087 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7596c9fb79-ffjfk_aa2f801e-0b13-4861-949e-3c2ef230b977/manager/0.log" Sep 09 21:06:07.947209 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.947174 2538 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:06:07.947567 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.947245 2538 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:06:07.947567 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.947262 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:06:07.947567 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:07.947276 2538 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" Sep 09 21:06:08.368861 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:08.368835 2538 scope.go:117] "RemoveContainer" containerID="54e9e6378739517d241a9fb5ac12b91cfe715d8a118736a75c9562ab4651ead7" Sep 09 21:06:08.369052 ip-10-0-143-6 kubenswrapper[2538]: E0909 21:06:08.369003 2538 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-p4p9p_llm(4ba47bf1-3ed3-4005-a265-7b12d74eab75)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-p4p9p" podUID="4ba47bf1-3ed3-4005-a265-7b12d74eab75" Sep 09 21:06:08.401194 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:08.401165 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-95gb5_56e44051-9885-4209-b5e2-c43d9598f9e7/payload-pre-processing/0.log" Sep 09 21:06:08.459605 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:08.459583 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d949964-m9n92_e4728dbe-431a-44c6-93e1-889c4e86b95e/router/0.log" Sep 09 21:06:09.692450 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:09.692425 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-95gb5_56e44051-9885-4209-b5e2-c43d9598f9e7/payload-pre-processing/0.log" Sep 09 21:06:09.730053 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:09.730029 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9d949964-m9n92_e4728dbe-431a-44c6-93e1-889c4e86b95e/router/0.log" Sep 09 21:06:13.908167 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:13.908135 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h64f5_fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040/kube-multus-additional-cni-plugins/0.log" Sep 09 21:06:13.933814 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:13.933792 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h64f5_fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040/egress-router-binary-copy/0.log" Sep 09 21:06:13.962646 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:13.962622 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h64f5_fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040/cni-plugins/0.log" Sep 09 21:06:13.988652 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:13.988632 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h64f5_fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040/bond-cni-plugin/0.log" Sep 09 21:06:14.012937 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:14.012919 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h64f5_fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040/routeoverride-cni/0.log" Sep 09 21:06:14.036888 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:14.036873 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h64f5_fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040/whereabouts-cni-bincopy/0.log" Sep 09 21:06:14.067311 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:14.067292 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h64f5_fcdd0f45-a57c-4b15-a2dc-7b20fa8ac040/whereabouts-cni/0.log" Sep 09 21:06:14.660738 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:14.660713 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rnlpl_76314433-6fa3-4bbe-835a-075283005f46/kube-multus/0.log" Sep 09 21:06:14.731683 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:14.731658 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-84ccl_b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e/network-metrics-daemon/0.log" Sep 09 21:06:14.753525 ip-10-0-143-6 kubenswrapper[2538]: I0909 21:06:14.753506 2538 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-84ccl_b6f32e5e-7e01-41aa-b6fd-fd2f272d3a4e/kube-rbac-proxy/0.log"