Sep 07 17:06:30.664141 ip-10-0-143-85 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 07 17:06:30.664151 ip-10-0-143-85 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 07 17:06:30.664160 ip-10-0-143-85 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 07 17:06:30.664427 ip-10-0-143-85 systemd[1]: Failed to start Kubernetes Kubelet. Sep 07 17:06:40.191041 ip-10-0-143-85 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 07 17:06:40.191056 ip-10-0-143-85 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 23a641315b8a461f9e4abc1f31ec04c1 -- Sep 07 17:08:10.604414 ip-10-0-143-85 systemd[1]: Starting Kubernetes Kubelet... Sep 07 17:08:11.061985 ip-10-0-143-85 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 17:08:11.061985 ip-10-0-143-85 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 07 17:08:11.061985 ip-10-0-143-85 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 17:08:11.061985 ip-10-0-143-85 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 07 17:08:11.061985 ip-10-0-143-85 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 17:08:11.064333 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.064241 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 07 17:08:11.069597 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069574 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 17:08:11.069597 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069594 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 17:08:11.069597 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069600 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069604 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069610 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069614 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069618 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069622 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069626 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069630 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069634 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069638 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069643 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069647 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069651 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069655 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069659 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069664 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069667 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069672 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069676 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 17:08:11.069754 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069679 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069683 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069687 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069692 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069699 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069705 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069709 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069713 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069716 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069720 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069723 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069726 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069729 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069733 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069736 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069739 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069741 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069744 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069748 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069751 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 17:08:11.070889 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069754 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069756 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069759 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069761 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069764 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069767 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069770 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069772 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069775 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069777 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069780 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069783 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069785 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069788 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069790 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069793 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069795 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069798 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069801 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069804 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 17:08:11.071430 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069806 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069809 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069812 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069814 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069817 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069820 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069823 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069825 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069828 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069831 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069833 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069836 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069838 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069841 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069843 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069847 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069851 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069853 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069856 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069858 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 17:08:11.071924 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069861 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069864 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069871 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069874 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069877 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069880 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069883 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069886 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069889 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069891 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069894 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069898 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069901 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069903 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069906 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069908 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069911 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.069914 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071478 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 17:08:11.072439 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071486 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071490 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071493 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071496 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071500 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071503 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071505 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071508 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071511 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071513 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071516 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071519 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071521 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071524 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071526 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071529 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071533 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071536 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071539 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071541 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 17:08:11.072893 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071544 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071547 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071549 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071553 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071556 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071559 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071561 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071564 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071567 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071569 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071572 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071575 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071578 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071580 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071583 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071586 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071588 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071591 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071594 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071597 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 17:08:11.073407 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071600 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071603 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071605 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071608 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071610 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071613 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071615 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071618 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071620 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071624 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071626 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071629 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071632 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071635 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071637 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071640 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071643 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071647 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071651 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071654 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 17:08:11.073907 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071657 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071660 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071663 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071665 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071667 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071670 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071673 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071675 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071677 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071680 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071682 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071685 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071687 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071690 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071692 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071694 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071697 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071699 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071702 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071704 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 17:08:11.074411 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071707 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071709 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071713 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071715 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071718 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071720 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071723 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071728 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071732 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071737 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071740 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071743 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071746 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071749 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071752 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071755 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071757 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.071760 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072172 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 17:08:11.074920 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072178 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072181 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072184 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072187 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072189 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072192 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072197 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072200 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072203 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072206 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072208 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072211 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072214 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072217 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072220 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072223 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072225 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072228 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072231 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 17:08:11.075404 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072233 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072237 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072240 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072243 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072245 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072248 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072251 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072253 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072256 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072258 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072261 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072263 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072266 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072268 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072271 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072274 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072276 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072278 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072281 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072284 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 17:08:11.075901 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072287 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072290 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072293 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072295 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072297 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072300 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072303 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072305 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072308 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072313 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072317 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072320 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072323 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072326 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072329 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072332 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072334 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072337 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072339 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 17:08:11.076412 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072341 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072344 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072346 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072349 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072351 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072354 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072357 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072359 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072362 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072364 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072367 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072369 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072372 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072374 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072377 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072380 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072383 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072385 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072388 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072391 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 17:08:11.076961 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072393 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072396 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072398 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072402 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072405 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072407 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072410 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072416 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072418 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072421 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072424 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072426 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072429 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072431 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072434 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072439 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072441 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072444 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072447 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.072450 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 17:08:11.077491 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072541 2563 flags.go:64] FLAG: --address="0.0.0.0" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072591 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072598 2563 flags.go:64] FLAG: --anonymous-auth="true" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072603 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072607 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072611 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072615 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072619 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072622 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072625 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072629 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072632 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072635 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072638 2563 flags.go:64] FLAG: --cgroup-root="" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072641 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072644 2563 flags.go:64] FLAG: --client-ca-file="" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072648 2563 flags.go:64] FLAG: --cloud-provider="external" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072652 2563 flags.go:64] FLAG: --cluster-dns="[]" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072656 2563 flags.go:64] FLAG: --cluster-domain="" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072661 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072664 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072667 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072671 2563 flags.go:64] FLAG: --container-log-max-files="5" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072675 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 07 17:08:11.077970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072678 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072681 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072684 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072688 2563 flags.go:64] FLAG: --contention-profiling="false" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072691 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072694 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072698 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072701 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072705 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072708 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072711 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072714 2563 flags.go:64] FLAG: --enable-load-reader="false" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072717 2563 flags.go:64] FLAG: --enable-server="true" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072720 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072724 2563 flags.go:64] FLAG: --event-burst="100" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072727 2563 flags.go:64] FLAG: --event-qps="50" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072730 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072734 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072736 2563 flags.go:64] FLAG: --eviction-hard="" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072740 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072743 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072746 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072750 2563 flags.go:64] FLAG: --eviction-soft="" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072753 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072756 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 07 17:08:11.078562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072759 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072762 2563 flags.go:64] FLAG: --experimental-mounter-path="" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072768 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072771 2563 flags.go:64] FLAG: --fail-swap-on="true" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072773 2563 flags.go:64] FLAG: --feature-gates="" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072777 2563 flags.go:64] FLAG: --file-check-frequency="20s" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072780 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072783 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072786 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072789 2563 flags.go:64] FLAG: --healthz-port="10248" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072793 2563 flags.go:64] FLAG: --help="false" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072796 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072799 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072802 2563 flags.go:64] FLAG: --http-check-frequency="20s" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072805 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072808 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072812 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072814 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072817 2563 flags.go:64] FLAG: --image-service-endpoint="" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072820 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072823 2563 flags.go:64] FLAG: --kube-api-burst="100" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072825 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072828 2563 flags.go:64] FLAG: --kube-api-qps="50" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072831 2563 flags.go:64] FLAG: --kube-reserved="" Sep 07 17:08:11.079202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072834 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072837 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072840 2563 flags.go:64] FLAG: --kubelet-cgroups="" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072843 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072846 2563 flags.go:64] FLAG: --lock-file="" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072849 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072852 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072855 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072861 2563 flags.go:64] FLAG: --log-json-split-stream="false" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072865 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072869 2563 flags.go:64] FLAG: --log-text-split-stream="false" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072872 2563 flags.go:64] FLAG: --logging-format="text" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072875 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072878 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072881 2563 flags.go:64] FLAG: --manifest-url="" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072884 2563 flags.go:64] FLAG: --manifest-url-header="" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072888 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072891 2563 flags.go:64] FLAG: --max-open-files="1000000" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072897 2563 flags.go:64] FLAG: --max-pods="110" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072900 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072903 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072906 2563 flags.go:64] FLAG: --memory-manager-policy="None" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072909 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072912 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072915 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 07 17:08:11.079783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072918 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072928 2563 flags.go:64] FLAG: --node-status-max-images="50" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072932 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072934 2563 flags.go:64] FLAG: --oom-score-adj="-999" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072937 2563 flags.go:64] FLAG: --pod-cidr="" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072940 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072946 2563 flags.go:64] FLAG: --pod-manifest-path="" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072948 2563 flags.go:64] FLAG: --pod-max-pids="-1" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072951 2563 flags.go:64] FLAG: --pods-per-core="0" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072954 2563 flags.go:64] FLAG: --port="10250" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072958 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072960 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c22d09bf89eedb72" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072964 2563 flags.go:64] FLAG: --qos-reserved="" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072966 2563 flags.go:64] FLAG: --read-only-port="10255" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072969 2563 flags.go:64] FLAG: --register-node="true" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072972 2563 flags.go:64] FLAG: --register-with-taints="" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072976 2563 flags.go:64] FLAG: --registry-burst="10" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072980 2563 flags.go:64] FLAG: --registry-qps="5" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072983 2563 flags.go:64] FLAG: --reserved-cpus="" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072986 2563 flags.go:64] FLAG: --reserved-memory="" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072990 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072993 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072996 2563 flags.go:64] FLAG: --rotate-certificates="false" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.072999 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073001 2563 flags.go:64] FLAG: --runonce="false" Sep 07 17:08:11.080467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073006 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073009 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073012 2563 flags.go:64] FLAG: --seccomp-default="false" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073014 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073017 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073020 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073023 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073027 2563 flags.go:64] FLAG: --storage-driver-password="root" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073029 2563 flags.go:64] FLAG: --storage-driver-secure="false" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073032 2563 flags.go:64] FLAG: --storage-driver-table="stats" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073035 2563 flags.go:64] FLAG: --storage-driver-user="root" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073038 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073041 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073044 2563 flags.go:64] FLAG: --system-cgroups="" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073047 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073052 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073054 2563 flags.go:64] FLAG: --tls-cert-file="" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073057 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073062 2563 flags.go:64] FLAG: --tls-min-version="" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073064 2563 flags.go:64] FLAG: --tls-private-key-file="" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073067 2563 flags.go:64] FLAG: --topology-manager-policy="none" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073070 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073073 2563 flags.go:64] FLAG: --topology-manager-scope="container" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073076 2563 flags.go:64] FLAG: --v="2" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073080 2563 flags.go:64] FLAG: --version="false" Sep 07 17:08:11.081105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073086 2563 flags.go:64] FLAG: --vmodule="" Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073089 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.073093 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073222 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073227 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073230 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073233 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073238 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073241 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073243 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073246 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073249 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073251 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073254 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073257 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073261 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073264 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073267 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073269 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 17:08:11.081743 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073272 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073274 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073276 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073279 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073282 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073284 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073287 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073289 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073291 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073294 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073297 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073301 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073304 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073309 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073312 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073315 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073318 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073320 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073323 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 17:08:11.082237 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073325 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073329 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073332 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073335 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073337 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073340 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073342 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073345 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073348 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073350 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073353 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073356 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073358 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073361 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073363 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073365 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073368 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073371 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073373 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073376 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 17:08:11.082698 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073378 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073381 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073383 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073387 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073389 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073391 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073396 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073398 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073401 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073404 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073407 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073411 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073414 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073418 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073421 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073424 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073426 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073429 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073431 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 17:08:11.083196 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073434 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073437 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073439 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073442 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073445 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073447 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073450 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073452 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073455 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073457 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073460 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073462 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073465 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073467 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073470 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073472 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073475 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073477 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073480 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073484 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 17:08:11.083674 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073486 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073489 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073492 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073495 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.073497 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.074232 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.082248 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.082415 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.083334 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.083345 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083423 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083430 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083435 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083440 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083444 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 17:08:11.084245 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083448 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083453 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083457 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083461 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083465 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083469 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083474 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083477 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083482 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083486 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083490 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083496 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083502 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083506 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083510 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083515 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083519 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083524 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083527 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 17:08:11.084649 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083533 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083538 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083542 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083545 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083549 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083553 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083557 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083561 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083565 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083569 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083573 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083576 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083580 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083585 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083590 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083595 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083599 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083603 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083607 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 17:08:11.085104 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083610 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083614 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083618 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083623 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083626 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083630 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083634 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083638 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083641 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083645 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083649 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083653 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083657 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083661 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083666 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083671 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083674 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083678 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083683 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 17:08:11.085577 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083686 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083690 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083694 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083698 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083702 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083706 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083710 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083714 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083718 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083722 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083726 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083729 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083733 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083737 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083741 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083744 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083748 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083752 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083756 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083760 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 17:08:11.086036 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083763 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083767 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083771 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083774 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083778 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083782 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083787 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083803 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083813 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083819 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083824 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083828 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083832 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083836 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083840 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083843 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 17:08:11.086539 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.083847 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.083855 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084015 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084021 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084025 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084029 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084033 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084037 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084041 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084045 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084049 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084053 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084057 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084060 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084064 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 17:08:11.086923 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084068 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084071 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084075 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084078 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084083 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084087 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084090 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084094 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084098 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084102 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084106 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084125 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084129 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084133 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084137 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084141 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084145 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084151 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084156 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084160 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 17:08:11.087323 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084164 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084168 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084173 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084177 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084181 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084184 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084189 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084192 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084196 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084200 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084204 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084208 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084212 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084216 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084219 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084223 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084227 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084231 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084235 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084239 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 17:08:11.087853 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084242 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084246 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084250 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084254 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084259 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084263 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084266 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084270 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084276 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084281 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084285 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084290 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084294 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084298 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084302 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084306 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084310 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084313 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084317 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 17:08:11.088357 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084321 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084325 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084329 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084333 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084337 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084341 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084344 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084347 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084351 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084356 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084359 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084363 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084368 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084374 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084377 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084380 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084384 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084389 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084393 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 17:08:11.089095 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084397 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084400 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084404 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084408 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084412 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084415 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084419 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:11.084423 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.084429 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.084572 2563 server.go:962] "Client rotation is on, will bootstrap in background" Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.087289 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.088391 2563 server.go:1019] "Starting client certificate rotation" Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.088513 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 17:08:11.089779 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.088566 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 17:08:11.091491 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.091467 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 17:08:11.092673 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.092655 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 17:08:11.104581 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.104560 2563 log.go:25] "Validated CRI v1 runtime API" Sep 07 17:08:11.109730 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.109710 2563 log.go:25] "Validated CRI v1 image API" Sep 07 17:08:11.110676 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.110652 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 07 17:08:11.114189 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.114168 2563 fs.go:135] Filesystem UUIDs: map[0b620744-5922-4472-991c-0fafc14181d1:/dev/nvme0n1p3 144d6f5e-d376-46f7-a6bf-a6c4b1490e2f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 07 17:08:11.114267 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.114186 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 07 17:08:11.118442 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.118284 2563 manager.go:216] Machine: {Timestamp:2026-09-07 17:08:11.117767282 +0000 UTC m=+0.384285455 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101097 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23974329cfef61075b95c68e50fa31 SystemUUID:ec239743-29cf-ef61-075b-95c68e50fa31 BootID:23a64131-5b8a-461f-9e4a-bc1f31ec04c1 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b1:85:2e:38:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b1:85:2e:38:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:02:56:4f:ce:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 07 17:08:11.118442 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.118433 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 07 17:08:11.118596 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.118560 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 07 17:08:11.121427 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.121409 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 17:08:11.122214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.122189 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 07 17:08:11.122363 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.122219 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-85.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 07 17:08:11.123082 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.123071 2563 topology_manager.go:138] "Creating topology manager with none policy" Sep 07 17:08:11.123138 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.123085 2563 container_manager_linux.go:309] "Creating device plugin manager" Sep 07 17:08:11.123138 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.123093 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 17:08:11.123138 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.123106 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 17:08:11.124935 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.124924 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 07 17:08:11.129343 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129325 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 07 17:08:11.129447 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129437 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 07 17:08:11.129493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129468 2563 kubelet.go:486] "Attempting to sync node with API server" Sep 07 17:08:11.129493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129478 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 07 17:08:11.129493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129489 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 07 17:08:11.129610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129499 2563 kubelet.go:398] "Adding apiserver pod source" Sep 07 17:08:11.129610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129508 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 07 17:08:11.129892 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129880 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 17:08:11.129926 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.129899 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 17:08:11.132571 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.132554 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 07 17:08:11.135566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135551 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 07 17:08:11.135637 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135581 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 07 17:08:11.135763 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135748 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 07 17:08:11.135824 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135774 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 07 17:08:11.135824 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135781 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 07 17:08:11.135824 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135786 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 07 17:08:11.135824 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135802 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 07 17:08:11.135824 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.135808 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 07 17:08:11.136383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136374 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 07 17:08:11.136383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136383 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 07 17:08:11.136460 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136389 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 07 17:08:11.136460 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136395 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 07 17:08:11.136460 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136403 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 07 17:08:11.136460 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136410 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 07 17:08:11.136460 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136434 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 07 17:08:11.136460 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.136444 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 07 17:08:11.141610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.141540 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:11.141610 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.141613 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 17:08:11.141783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.141645 2563 server.go:1268] "Started kubelet" Sep 07 17:08:11.141783 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.141707 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 17:08:11.141885 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.141753 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 07 17:08:11.141885 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.141823 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 07 17:08:11.141962 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.141926 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 07 17:08:11.142652 ip-10-0-143-85 systemd[1]: Started Kubernetes Kubelet. Sep 07 17:08:11.143258 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.143089 2563 server.go:310] "Adding debug handlers to kubelet server" Sep 07 17:08:11.143821 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.143803 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 07 17:08:11.146779 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.146755 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 07 17:08:11.149275 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.149258 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 07 17:08:11.149797 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.148671 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eeda65e61a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.141613082 +0000 UTC m=+0.408131303,LastTimestamp:2026-09-07 17:08:11.141613082 +0000 UTC m=+0.408131303,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.149899 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.149876 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 07 17:08:11.150123 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.150086 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:11.150688 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.150673 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 07 17:08:11.150688 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.150675 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 07 17:08:11.150688 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.150698 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 07 17:08:11.150856 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.150782 2563 reconstruct.go:97] "Volume reconstruction finished" Sep 07 17:08:11.150856 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.150788 2563 reconciler.go:29] "Reconciler: start to sync state" Sep 07 17:08:11.152310 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.152292 2563 factory.go:55] Registering systemd factory Sep 07 17:08:11.152391 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.152314 2563 factory.go:223] Registration of the systemd container factory successfully Sep 07 17:08:11.152552 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.152535 2563 factory.go:153] Registering CRI-O factory Sep 07 17:08:11.152552 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.152553 2563 factory.go:223] Registration of the crio container factory successfully Sep 07 17:08:11.152689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.152605 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 07 17:08:11.152689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.152623 2563 factory.go:103] Registering Raw factory Sep 07 17:08:11.152689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.152639 2563 manager.go:1196] Started watching for new ooms in manager Sep 07 17:08:11.153015 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.153000 2563 manager.go:349] Starting recovery of all containers Sep 07 17:08:11.155580 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.155550 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 07 17:08:11.155697 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.155675 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 07 17:08:11.165243 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.165219 2563 manager.go:354] Recovery completed Sep 07 17:08:11.171189 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.171174 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.173390 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.173363 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.173490 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.173399 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.173490 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.173411 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.174001 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.173985 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 07 17:08:11.174001 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.173998 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 07 17:08:11.174156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.174016 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 07 17:08:11.174156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.174016 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.174777 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.174762 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.174853 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.174791 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.174853 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.174805 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.175265 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.175178 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.176257 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.176242 2563 policy_none.go:49] "None policy: Start" Sep 07 17:08:11.176312 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.176264 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 07 17:08:11.176312 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.176274 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 07 17:08:11.178252 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.178240 2563 policy_none.go:47] "Start" Sep 07 17:08:11.188617 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.188598 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9ttgj" Sep 07 17:08:11.189053 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.188981 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.201462 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.201384 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b26d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,LastTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.208744 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.208647 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4ac27b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.174775256 +0000 UTC m=+0.441293435,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.218516 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b0056\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.174797474 +0000 UTC m=+0.441315649,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.225244 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.226436 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.226485 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.226520 2563 kubelet.go:2448] "Starting kubelet main sync loop" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.226621 2563 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.229244 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.229203 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b26d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b26d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,LastTimestamp:2026-09-07 17:08:11.174810634 +0000 UTC m=+0.441328808,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.232741 2563 manager.go:337] "Starting Device Plugin manager" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.232787 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.232797 2563 server.go:87] "Starting device plugin registration server" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.233011 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.233019 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.233140 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.233212 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.233220 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.233675 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 07 17:08:11.235025 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.233737 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:11.236697 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.236632 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedff4f525 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.234874661 +0000 UTC m=+0.501392825,LastTimestamp:2026-09-07 17:08:11.234874661 +0000 UTC m=+0.501392825,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.327484 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.327381 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal"] Sep 07 17:08:11.327484 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.327484 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.329307 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.329289 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.329418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.329326 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.329418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.329339 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.331642 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.331626 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.331764 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.331742 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.331831 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.331781 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.332440 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.332422 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.332510 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.332455 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.332510 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.332462 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.332510 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.332488 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.332510 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.332503 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.332679 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.332469 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.333139 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.333106 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.333924 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.333902 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.333999 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.333929 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.333999 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.333939 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.333999 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.333970 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.334885 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.334870 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.334933 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.334904 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.335548 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.335533 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.335625 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.335563 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.335625 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.335573 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.340557 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.340460 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4ac27b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.329307693 +0000 UTC m=+0.595825868,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.347603 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.347534 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b0056\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.329332273 +0000 UTC m=+0.595850448,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.347697 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.347591 2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.350216 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.350149 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b26d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b26d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,LastTimestamp:2026-09-07 17:08:11.329344427 +0000 UTC m=+0.595862603,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.351761 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.351745 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/778133e8e3dd073d85d9bb9b749e3d88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal\" (UID: \"778133e8e3dd073d85d9bb9b749e3d88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.351813 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.351767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/778133e8e3dd073d85d9bb9b749e3d88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal\" (UID: \"778133e8e3dd073d85d9bb9b749e3d88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.351813 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.351784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5d8b8706f17884ecb94f9bab542e98f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-85.ec2.internal\" (UID: \"f5d8b8706f17884ecb94f9bab542e98f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.359140 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.359106 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Sep 07 17:08:11.359216 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.359138 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4ac27b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.332440155 +0000 UTC m=+0.598958329,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.361975 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.361956 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.366652 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.366635 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.369270 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.369209 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b0056\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.332461937 +0000 UTC m=+0.598980111,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.378978 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.378907 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4ac27b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.332475895 +0000 UTC m=+0.598994074,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.389555 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.389480 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b0056\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.332493927 +0000 UTC m=+0.599012105,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.395987 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.395925 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b26d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b26d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,LastTimestamp:2026-09-07 17:08:11.332508479 +0000 UTC m=+0.599026657,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.409929 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.409852 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b26d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b26d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,LastTimestamp:2026-09-07 17:08:11.332522688 +0000 UTC m=+0.599040865,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.421269 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.421196 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4ac27b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.333915662 +0000 UTC m=+0.600433836,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.431172 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.431075 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b0056\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.333934322 +0000 UTC m=+0.600452497,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.442044 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.441971 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b26d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b26d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,LastTimestamp:2026-09-07 17:08:11.33394265 +0000 UTC m=+0.600460825,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.452611 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.452588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/778133e8e3dd073d85d9bb9b749e3d88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal\" (UID: \"778133e8e3dd073d85d9bb9b749e3d88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.452737 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.452615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/778133e8e3dd073d85d9bb9b749e3d88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal\" (UID: \"778133e8e3dd073d85d9bb9b749e3d88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.452737 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.452631 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5d8b8706f17884ecb94f9bab542e98f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-85.ec2.internal\" (UID: \"f5d8b8706f17884ecb94f9bab542e98f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.452737 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.452698 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5d8b8706f17884ecb94f9bab542e98f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-85.ec2.internal\" (UID: \"f5d8b8706f17884ecb94f9bab542e98f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.452737 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.452698 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/778133e8e3dd073d85d9bb9b749e3d88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal\" (UID: \"778133e8e3dd073d85d9bb9b749e3d88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.452737 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.452698 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/778133e8e3dd073d85d9bb9b749e3d88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal\" (UID: \"778133e8e3dd073d85d9bb9b749e3d88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.454205 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.454139 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4ac27b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.335547407 +0000 UTC m=+0.602065584,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.461641 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.461571 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b0056\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.335568251 +0000 UTC m=+0.602086424,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.471079 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.471010 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b26d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b26d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.17341461 +0000 UTC m=+0.439932784,LastTimestamp:2026-09-07 17:08:11.335577431 +0000 UTC m=+0.602095604,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.547820 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.547783 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:11.551335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.551313 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:11.551406 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.551354 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:11.551406 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.551368 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:11.551406 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:11.551395 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.561000 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.560902 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4ac27b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4ac27b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173388923 +0000 UTC m=+0.439907103,LastTimestamp:2026-09-07 17:08:11.551335878 +0000 UTC m=+0.817854059,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:11.567402 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.567377 2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:11.567467 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:11.567345 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-85.ec2.internal.18d318eedc4b0056\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-85.ec2.internal.18d318eedc4b0056 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-85.ec2.internal,UID:ip-10-0-143-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:11.173404758 +0000 UTC m=+0.439922932,LastTimestamp:2026-09-07 17:08:11.551361889 +0000 UTC m=+0.817880063,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:12.858163 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:12.858125 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:12.864493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:12.864130 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" Sep 07 17:08:12.954659 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:12.954628 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms" Sep 07 17:08:13.161737 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.161652 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:13.162593 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.162573 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:13.162685 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.162605 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:13.162685 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.162617 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:13.162685 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.162644 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:13.184129 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.184093 2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:13.273476 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.273439 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 07 17:08:13.345191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.345016 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:13.407699 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.407668 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 07 17:08:13.521135 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:13.521094 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod778133e8e3dd073d85d9bb9b749e3d88.slice/crio-bc768d841b9383a49de7c06f22b3377a627c05c65bfb8de469ebd1d34200ead4 WatchSource:0}: Error finding container bc768d841b9383a49de7c06f22b3377a627c05c65bfb8de469ebd1d34200ead4: Status 404 returned error can't find the container with id bc768d841b9383a49de7c06f22b3377a627c05c65bfb8de469ebd1d34200ead4 Sep 07 17:08:13.525662 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.525647 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 17:08:13.534743 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.534651 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318ef68822f32 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal,UID:778133e8e3dd073d85d9bb9b749e3d88,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\",Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:13.525831474 +0000 UTC m=+1.598732785,LastTimestamp:2026-09-07 17:08:13.525831474 +0000 UTC m=+1.598732785,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:13.541121 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.541102 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:13.542103 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.542084 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:13.542180 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.542122 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:13.542180 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.542136 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:13.544286 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.544214 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-85.ec2.internal.18d318ef688307b7 kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-85.ec2.internal,UID:f5d8b8706f17884ecb94f9bab542e98f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\",Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:13.525886903 +0000 UTC m=+1.598788215,LastTimestamp:2026-09-07 17:08:13.525886903 +0000 UTC m=+1.598788215,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:13.765945 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.765859 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s" Sep 07 17:08:13.887877 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.887840 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 17:08:13.913520 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:13.913485 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 17:08:13.984925 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.984890 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:13.986152 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.986127 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:13.986278 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.986177 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:13.986278 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.986195 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:13.986278 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:13.986228 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:14.003351 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:14.003318 2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:14.345657 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:14.345621 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:14.426214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:14.426155 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" event={"ID":"778133e8e3dd073d85d9bb9b749e3d88","Type":"ContainerStarted","Data":"bc768d841b9383a49de7c06f22b3377a627c05c65bfb8de469ebd1d34200ead4"} Sep 07 17:08:14.427519 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:14.427476 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" event={"ID":"f5d8b8706f17884ecb94f9bab542e98f","Type":"ContainerStarted","Data":"e043b3764b32fa76d16a1ab95f002da226841f8f3ceda802579bf8672e4ebc91"} Sep 07 17:08:15.345572 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.345537 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:15.345572 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.345540 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 07 17:08:15.375582 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.375545 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s" Sep 07 17:08:15.438005 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.437922 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-85.ec2.internal.18d318efd9ea97f5 kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-85.ec2.internal,UID:f5d8b8706f17884ecb94f9bab542e98f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\" in 1.902s (1.902s including waiting). Image size: 321671430 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.428499445 +0000 UTC m=+3.501400779,LastTimestamp:2026-09-07 17:08:15.428499445 +0000 UTC m=+3.501400779,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:15.447211 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.447094 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318efda010498 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal,UID:778133e8e3dd073d85d9bb9b749e3d88,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" in 1.904s (1.904s including waiting). Image size: 282837706 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.429969048 +0000 UTC m=+3.502870377,LastTimestamp:2026-09-07 17:08:15.429969048 +0000 UTC m=+3.502870377,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:15.516247 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.516148 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-85.ec2.internal.18d318efdea81e5c kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-85.ec2.internal,UID:f5d8b8706f17884ecb94f9bab542e98f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.50802902 +0000 UTC m=+3.580930352,LastTimestamp:2026-09-07 17:08:15.50802902 +0000 UTC m=+3.580930352,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:15.525665 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.525584 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-85.ec2.internal.18d318efdf24734d kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-85.ec2.internal,UID:f5d8b8706f17884ecb94f9bab542e98f,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.516177229 +0000 UTC m=+3.589078563,LastTimestamp:2026-09-07 17:08:15.516177229 +0000 UTC m=+3.589078563,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:15.604102 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.604007 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:15.605166 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.605128 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:15.605302 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.605173 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:15.605302 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.605190 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:15.605302 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.605236 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:15.621947 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.621916 2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:15.622151 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.622117 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 07 17:08:15.646391 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.646365 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:15.647459 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.647434 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:15.647645 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.647478 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:15.647645 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:15.647491 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:15.692699 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.692601 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318efe9244dc8 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal,UID:778133e8e3dd073d85d9bb9b749e3d88,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.683939784 +0000 UTC m=+3.756841108,LastTimestamp:2026-09-07 17:08:15.683939784 +0000 UTC m=+3.756841108,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:15.702796 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:15.702700 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318efe9a7ffb4 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal,UID:778133e8e3dd073d85d9bb9b749e3d88,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.692570548 +0000 UTC m=+3.765471882,LastTimestamp:2026-09-07 17:08:15.692570548 +0000 UTC m=+3.765471882,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:16.346151 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.346118 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:16.409465 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:16.409436 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 17:08:16.431549 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.431515 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" event={"ID":"f5d8b8706f17884ecb94f9bab542e98f","Type":"ContainerStarted","Data":"f92063a25b5d3f6d6a115ae02ba0faf8e734b532bf0102c4a3fa136c1927e70a"} Sep 07 17:08:16.431708 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.431530 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:16.432560 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.432537 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:16.432662 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.432576 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:16.432662 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.432590 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:16.435227 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:16.435202 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:16.435487 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.435293 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" event={"ID":"778133e8e3dd073d85d9bb9b749e3d88","Type":"ContainerStarted","Data":"1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d"} Sep 07 17:08:16.435543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.435505 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:16.436443 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.436423 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:16.436547 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.436460 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:16.436547 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:16.436474 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:16.436679 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:16.436664 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:17.118956 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:17.118916 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 17:08:17.357473 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.357384 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:17.436437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.436412 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.436412 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.437279 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.437302 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.437312 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.437281 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.437350 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:17.437363 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:17.437512 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:17.441703 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:17.437608 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:18.345027 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:18.344994 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:18.586580 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:18.586544 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s" Sep 07 17:08:18.822945 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:18.822842 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:18.827421 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:18.827400 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:18.827541 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:18.827439 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:18.827541 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:18.827454 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:18.827541 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:18.827488 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:18.847923 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:18.847890 2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:19.345493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:19.345462 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:20.123319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:20.123278 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:20.124315 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:20.124298 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:20.124383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:20.124329 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:20.124383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:20.124339 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:20.344430 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:20.344399 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:20.570459 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:20.570369 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 07 17:08:20.570459 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:20.570369 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 07 17:08:21.343584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:21.343512 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:22.139213 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:22.139180 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 17:08:22.254627 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:22.254596 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 17:08:22.345273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:22.345240 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:22.428042 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:22.427956 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:23.346484 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:23.346461 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:24.344425 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:24.344393 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:24.994252 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:24.994212 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 07 17:08:25.248083 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:25.247992 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:25.249002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:25.248983 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:25.249110 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:25.249012 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:25.249110 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:25.249022 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:25.249110 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:25.249056 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:25.267257 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:25.267230 2563 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:25.344341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:25.344308 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:26.347833 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:26.347799 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:27.346743 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.346541 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:27.451399 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.451370 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal_778133e8e3dd073d85d9bb9b749e3d88/setup/0.log" Sep 07 17:08:27.451768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.451409 2563 generic.go:356] "Generic (PLEG): container finished" podID="778133e8e3dd073d85d9bb9b749e3d88" containerID="1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d" exitCode=1 Sep 07 17:08:27.451768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.451443 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" event={"ID":"778133e8e3dd073d85d9bb9b749e3d88","Type":"ContainerDied","Data":"1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d"} Sep 07 17:08:27.451768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.451545 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:27.452347 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.452329 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:27.452389 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.452360 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:27.452389 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:27.452370 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:27.452550 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:27.452538 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:27.463279 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:27.463199 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318f2a6c8c79d openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal,UID:778133e8e3dd073d85d9bb9b749e3d88,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:27.455547293 +0000 UTC m=+15.528448630,LastTimestamp:2026-09-07 17:08:27.455547293 +0000 UTC m=+15.528448630,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:27.552527 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:27.552441 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318efe9244dc8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318efe9244dc8 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal,UID:778133e8e3dd073d85d9bb9b749e3d88,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.683939784 +0000 UTC m=+3.756841108,LastTimestamp:2026-09-07 17:08:27.542765026 +0000 UTC m=+15.615666353,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:27.562439 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:27.562361 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318efe9a7ffb4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal.18d318efe9a7ffb4 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal,UID:778133e8e3dd073d85d9bb9b749e3d88,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-143-85.ec2.internal,},FirstTimestamp:2026-09-07 17:08:15.692570548 +0000 UTC m=+3.765471882,LastTimestamp:2026-09-07 17:08:27.552026032 +0000 UTC m=+15.624927367,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-85.ec2.internal,}" Sep 07 17:08:28.155849 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:28.155814 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 07 17:08:28.347031 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.346983 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 17:08:28.455034 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.454958 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal_778133e8e3dd073d85d9bb9b749e3d88/setup/0.log" Sep 07 17:08:28.455034 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.455010 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" event={"ID":"778133e8e3dd073d85d9bb9b749e3d88","Type":"ContainerStarted","Data":"34ef9cd6f7d635b6a3b836f7c3822a27ed03e727d438a128e0dbd32bae9ec3dd"} Sep 07 17:08:28.455456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.455144 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:28.456042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.456027 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:28.456131 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.456059 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:28.456131 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.456076 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:28.456306 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:28.456292 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:28.921341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:28.921306 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9ttgj" Sep 07 17:08:29.020786 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.020722 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:29.022694 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.022675 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:29.022792 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.022710 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:29.022792 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.022724 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:29.281616 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.281587 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 07 17:08:29.358039 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.358006 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.375614 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.375593 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.432781 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.432758 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.704182 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.704100 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.704182 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:29.704143 2563 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.744245 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.744213 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.761537 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.761506 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.825608 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.825583 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:29.926191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.926147 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 17:03:28 +0000 UTC" deadline="2028-03-06 04:09:13.556907238 +0000 UTC" Sep 07 17:08:29.926191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:29.926186 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13091h0m43.630723791s" Sep 07 17:08:30.093477 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:30.093450 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:30.093477 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:30.093476 2563 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:30.347850 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:30.347793 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:30.363419 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:30.363399 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:30.419803 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:30.419784 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:30.699753 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:30.699661 2563 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:30.699753 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:30.699686 2563 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-143-85.ec2.internal" not found Sep 07 17:08:32.008497 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.008461 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-85.ec2.internal\" not found" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:32.267471 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.267364 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 17:08:32.268435 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.268419 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 17:08:32.268505 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.268451 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 17:08:32.268505 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.268466 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 17:08:32.268505 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.268497 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:32.283011 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.282988 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-85.ec2.internal" Sep 07 17:08:32.283011 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.283014 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-85.ec2.internal\": node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.298459 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.298432 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.399573 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.399532 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.428909 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.428880 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.499880 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.499844 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.531127 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.531076 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 17:08:32.600180 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.600148 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.700899 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.700871 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.801600 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.801532 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.902217 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:32.902177 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:32.914014 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:32.913990 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 17:08:33.002524 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.002495 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.103029 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.102959 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.203533 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.203500 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.304148 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.304113 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.405021 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.404957 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.505715 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.505671 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.606210 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.606159 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.706768 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.706682 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.807293 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:33.807258 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-85.ec2.internal\" not found" Sep 07 17:08:33.839384 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:33.839364 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 17:08:33.844267 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:33.844250 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" Sep 07 17:08:33.855174 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:33.855155 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 17:08:33.856178 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:33.856164 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" Sep 07 17:08:33.864789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:33.864771 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 17:08:34.335549 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.335519 2563 apiserver.go:52] "Watching apiserver" Sep 07 17:08:34.341438 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.341405 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 07 17:08:34.342613 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.342593 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc","openshift-cluster-node-tuning-operator/tuned-nxwnb","openshift-dns/node-resolver-r97tb","openshift-image-registry/node-ca-bgsdc","openshift-multus/multus-zgbmj","openshift-multus/network-metrics-daemon-cx8hf","kube-system/global-pull-secret-syncer-nmmvg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal","openshift-multus/multus-additional-cni-plugins-dz454","openshift-network-diagnostics/network-check-target-vmndt","openshift-network-operator/iptables-alerter-ps5c8","openshift-ovn-kubernetes/ovnkube-node-hfwh8","kube-system/konnectivity-agent-6ngms","kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal"] Sep 07 17:08:34.345030 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.345013 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.345102 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.345084 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:34.347141 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.347120 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.349187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.349168 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.349387 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.349360 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6s8dq\"" Sep 07 17:08:34.349387 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.349361 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 07 17:08:34.349560 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.349403 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 07 17:08:34.351209 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.351193 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.351800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.351784 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 07 17:08:34.352002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.351988 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bbvbx\"" Sep 07 17:08:34.352435 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.352418 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 07 17:08:34.352995 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.352979 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 07 17:08:34.353081 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.353013 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sjvkg\"" Sep 07 17:08:34.353081 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.353068 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 07 17:08:34.353742 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.353448 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 07 17:08:34.353742 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.353523 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.355036 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.355012 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 07 17:08:34.355108 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.355016 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 07 17:08:34.355415 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.355397 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 07 17:08:34.355415 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.355407 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 07 17:08:34.355554 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.355472 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7q2tx\"" Sep 07 17:08:34.355600 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.355554 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 07 17:08:34.355741 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.355710 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:34.355810 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.355788 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:34.358358 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.358339 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.361268 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.360356 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 07 17:08:34.362381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.361452 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 07 17:08:34.362381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.361931 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w6zsh\"" Sep 07 17:08:34.362381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.362154 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.362381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.362236 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9e168995-8f48-4329-b17f-e1f39943efc8-agent-certs\") pod \"konnectivity-agent-6ngms\" (UID: \"9e168995-8f48-4329-b17f-e1f39943efc8\") " pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.362381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.362273 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9e168995-8f48-4329-b17f-e1f39943efc8-konnectivity-ca\") pod \"konnectivity-agent-6ngms\" (UID: \"9e168995-8f48-4329-b17f-e1f39943efc8\") " pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.362381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.362334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d40dd6a8-670a-447b-9d18-7ec8645d13ef-kubelet-config\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.363310 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.362940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d40dd6a8-670a-447b-9d18-7ec8645d13ef-dbus\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.363310 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.363011 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.363986 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.363966 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 07 17:08:34.364281 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.364266 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 07 17:08:34.364368 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.364299 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pm6bs\"" Sep 07 17:08:34.364368 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.364304 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 07 17:08:34.364563 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.364547 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.366414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.366281 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 07 17:08:34.366414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.366287 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 07 17:08:34.366414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.366287 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f2bv9\"" Sep 07 17:08:34.366857 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.366838 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 17:08:34.367579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.367561 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:34.367672 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.367623 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:34.369825 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.369805 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.372406 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.372390 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.373661 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.373645 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rjcsb\"" Sep 07 17:08:34.373761 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.373681 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 07 17:08:34.373810 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.373790 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 07 17:08:34.375114 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.375100 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 07 17:08:34.375471 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.375453 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 07 17:08:34.375619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.375603 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 07 17:08:34.375860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.375706 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2mbcw\"" Sep 07 17:08:34.375860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.375712 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 07 17:08:34.375860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.375782 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 07 17:08:34.376044 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.376023 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 07 17:08:34.376079 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.376040 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 07 17:08:34.393570 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.393553 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pzvtj" Sep 07 17:08:34.401524 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.401507 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pzvtj" Sep 07 17:08:34.445092 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.445061 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 07 17:08:34.463240 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xchs9\" (UniqueName: \"kubernetes.io/projected/66832598-bf55-4875-9c98-0b96b2582b7c-kube-api-access-xchs9\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.463240 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463241 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-node-log\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463296 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463325 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463342 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-lib-modules\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463358 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-slash\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-var-lib-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463393 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vrq\" (UniqueName: \"kubernetes.io/projected/b29aaeeb-cf84-4293-b329-32de1e621742-kube-api-access-z5vrq\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-conf-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-device-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-cni-bin\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36380f08-07a9-4f59-9a84-8a639a5943cd-hosts-file\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-hostroot\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463501 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-daemon-config\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463516 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9e168995-8f48-4329-b17f-e1f39943efc8-agent-certs\") pod \"konnectivity-agent-6ngms\" (UID: \"9e168995-8f48-4329-b17f-e1f39943efc8\") " pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-log-socket\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463567 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-ovnkube-script-lib\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-registration-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463614 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02394e61-3140-406c-afa3-84ffc266a641-host\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463630 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-system-cni-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463660 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-host\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463693 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66832598-bf55-4875-9c98-0b96b2582b7c-host-slash\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463758 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-ovn\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463789 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-env-overrides\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463817 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d40dd6a8-670a-447b-9d18-7ec8645d13ef-kubelet-config\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-system-cni-dir\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.463884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-systemd\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-tmp\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463901 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d40dd6a8-670a-447b-9d18-7ec8645d13ef-kubelet-config\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463909 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-cni-netd\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463936 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36380f08-07a9-4f59-9a84-8a639a5943cd-tmp-dir\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.463987 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464031 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysctl-conf\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464055 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-tuned\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.464064 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464082 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-cni-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464114 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-socket-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.464132 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret podName:d40dd6a8-670a-447b-9d18-7ec8645d13ef nodeName:}" failed. No retries permitted until 2026-09-07 17:08:34.964098394 +0000 UTC m=+23.036999704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret") pod "global-pull-secret-syncer-nmmvg" (UID: "d40dd6a8-670a-447b-9d18-7ec8645d13ef") : object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmt8k\" (UniqueName: \"kubernetes.io/projected/02394e61-3140-406c-afa3-84ffc266a641-kube-api-access-rmt8k\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b29aaeeb-cf84-4293-b329-32de1e621742-ovn-node-metrics-cert\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464224 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96f76600-5d34-4109-8f07-3b7fa97658a4-cni-binary-copy\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.464437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464238 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d40dd6a8-670a-447b-9d18-7ec8645d13ef-dbus\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464287 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysctl-d\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-var-lib-kubelet\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464313 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464338 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-k8s-cni-cncf-io\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464375 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-multus-certs\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464398 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d40dd6a8-670a-447b-9d18-7ec8645d13ef-dbus\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx7dq\" (UniqueName: \"kubernetes.io/projected/bae269a1-2dfb-42d8-85ef-47939480e7ff-kube-api-access-qx7dq\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-modprobe-d\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-etc-kubernetes\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464941 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66832598-bf55-4875-9c98-0b96b2582b7c-iptables-alerter-script\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464967 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-run-netns\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.464989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-netns\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465052 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02394e61-3140-406c-afa3-84ffc266a641-serviceca\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nzfj\" (UniqueName: \"kubernetes.io/projected/c9a30523-229a-47bb-8e6e-c7f2b20fd301-kube-api-access-9nzfj\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.465184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-kubernetes\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465123 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfxrp\" (UniqueName: \"kubernetes.io/projected/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-kube-api-access-gfxrp\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-systemd-units\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465207 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-run-ovn-kubernetes\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465231 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snvhp\" (UniqueName: \"kubernetes.io/projected/96f76600-5d34-4109-8f07-3b7fa97658a4-kube-api-access-snvhp\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465252 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cnibin\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465304 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-run\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465323 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-systemd\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9e168995-8f48-4329-b17f-e1f39943efc8-konnectivity-ca\") pod \"konnectivity-agent-6ngms\" (UID: \"9e168995-8f48-4329-b17f-e1f39943efc8\") " pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-ovnkube-config\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-cnibin\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-cni-bin\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-cni-multus\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6jxx\" (UniqueName: \"kubernetes.io/projected/d590b05d-a3ad-434d-bc55-eb1f23c0e214-kube-api-access-l6jxx\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-kubelet\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.465789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-etc-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-kubelet\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465581 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-sys-fs\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465626 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465656 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysconfig\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q7h4\" (UniqueName: \"kubernetes.io/projected/36380f08-07a9-4f59-9a84-8a639a5943cd-kube-api-access-7q7h4\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465762 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-socket-dir-parent\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-sys\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.466273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.465889 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9e168995-8f48-4329-b17f-e1f39943efc8-konnectivity-ca\") pod \"konnectivity-agent-6ngms\" (UID: \"9e168995-8f48-4329-b17f-e1f39943efc8\") " pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.468231 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.468214 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9e168995-8f48-4329-b17f-e1f39943efc8-agent-certs\") pod \"konnectivity-agent-6ngms\" (UID: \"9e168995-8f48-4329-b17f-e1f39943efc8\") " pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.512419 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.512355 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-85.ec2.internal" podStartSLOduration=1.512339326 podStartE2EDuration="1.512339326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:34.512076844 +0000 UTC m=+22.584978177" watchObservedRunningTime="2026-09-07 17:08:34.512339326 +0000 UTC m=+22.585240658" Sep 07 17:08:34.566162 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566135 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.566254 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-lib-modules\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.566254 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566202 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-slash\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.566254 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-var-lib-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.566254 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vrq\" (UniqueName: \"kubernetes.io/projected/b29aaeeb-cf84-4293-b329-32de1e621742-kube-api-access-z5vrq\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566275 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-conf-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566301 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-device-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566326 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-cni-bin\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566330 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-lib-modules\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566363 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-var-lib-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566376 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-cni-bin\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566374 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-conf-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566333 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-slash\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566396 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-device-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36380f08-07a9-4f59-9a84-8a639a5943cd-hosts-file\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-hostroot\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566502 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-daemon-config\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36380f08-07a9-4f59-9a84-8a639a5943cd-hosts-file\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566526 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.566543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566551 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-log-socket\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566579 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-log-socket\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566587 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-ovnkube-script-lib\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566613 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-registration-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566641 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02394e61-3140-406c-afa3-84ffc266a641-host\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566627 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566667 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-registration-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566552 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-hostroot\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566666 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-system-cni-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566713 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-host\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566759 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566768 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66832598-bf55-4875-9c98-0b96b2582b7c-host-slash\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02394e61-3140-406c-afa3-84ffc266a641-host\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-ovn\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566762 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-host\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-system-cni-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566822 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66832598-bf55-4875-9c98-0b96b2582b7c-host-slash\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.567223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-env-overrides\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-system-cni-dir\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566871 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-ovn\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566883 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-systemd\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-tmp\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-cni-netd\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566926 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-system-cni-dir\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36380f08-07a9-4f59-9a84-8a639a5943cd-tmp-dir\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.566985 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysctl-conf\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567008 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-tuned\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567030 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-cni-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567053 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-socket-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmt8k\" (UniqueName: \"kubernetes.io/projected/02394e61-3140-406c-afa3-84ffc266a641-kube-api-access-rmt8k\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567101 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b29aaeeb-cf84-4293-b329-32de1e621742-ovn-node-metrics-cert\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567151 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96f76600-5d34-4109-8f07-3b7fa97658a4-cni-binary-copy\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:34.568196 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysctl-d\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567223 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-daemon-config\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567246 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567284 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-cni-netd\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-var-lib-kubelet\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-systemd\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-k8s-cni-cncf-io\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567375 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-multus-certs\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx7dq\" (UniqueName: \"kubernetes.io/projected/bae269a1-2dfb-42d8-85ef-47939480e7ff-kube-api-access-qx7dq\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567426 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-modprobe-d\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-etc-kubernetes\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567466 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36380f08-07a9-4f59-9a84-8a639a5943cd-tmp-dir\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66832598-bf55-4875-9c98-0b96b2582b7c-iptables-alerter-script\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-run-netns\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567524 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-netns\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567550 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567577 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02394e61-3140-406c-afa3-84ffc266a641-serviceca\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.569019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567602 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nzfj\" (UniqueName: \"kubernetes.io/projected/c9a30523-229a-47bb-8e6e-c7f2b20fd301-kube-api-access-9nzfj\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567625 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysctl-conf\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567630 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-kubernetes\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567656 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfxrp\" (UniqueName: \"kubernetes.io/projected/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-kube-api-access-gfxrp\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567681 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-systemd-units\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567708 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-run-ovn-kubernetes\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snvhp\" (UniqueName: \"kubernetes.io/projected/96f76600-5d34-4109-8f07-3b7fa97658a4-kube-api-access-snvhp\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cnibin\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567837 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-run\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567838 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-env-overrides\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567865 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-systemd\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567893 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-ovnkube-config\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567912 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567918 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-cnibin\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567960 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-cni-bin\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567969 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-cnibin\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.569848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.567988 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-modprobe-d\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-cni-multus\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568038 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-var-lib-kubelet\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6jxx\" (UniqueName: \"kubernetes.io/projected/d590b05d-a3ad-434d-bc55-eb1f23c0e214-kube-api-access-l6jxx\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568070 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-k8s-cni-cncf-io\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568078 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-kubelet\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568101 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-etc-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568103 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-multus-certs\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-kubelet\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568148 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-sys-fs\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysconfig\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568224 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q7h4\" (UniqueName: \"kubernetes.io/projected/36380f08-07a9-4f59-9a84-8a639a5943cd-kube-api-access-7q7h4\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-socket-dir-parent\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568306 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-run-ovn-kubernetes\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568309 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-sys\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.570546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xchs9\" (UniqueName: \"kubernetes.io/projected/66832598-bf55-4875-9c98-0b96b2582b7c-kube-api-access-xchs9\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568364 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-sys\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-node-log\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568393 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568408 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-etc-kubernetes\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568466 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-run-netns\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568651 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96f76600-5d34-4109-8f07-3b7fa97658a4-cni-binary-copy\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.568777 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568896 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysctl-d\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568898 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02394e61-3140-406c-afa3-84ffc266a641-serviceca\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.568971 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs podName:d590b05d-a3ad-434d-bc55-eb1f23c0e214 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:35.06895313 +0000 UTC m=+23.141854455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs") pod "network-metrics-daemon-cx8hf" (UID: "d590b05d-a3ad-434d-bc55-eb1f23c0e214") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.568949 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569128 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-cni-bin\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569161 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-cni-multus\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569227 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-cni-dir\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569283 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-multus-socket-dir-parent\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569321 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-kubernetes\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.571042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569364 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569374 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-node-log\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569406 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-systemd-units\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569429 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-kubelet\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569473 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-etc-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569507 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96f76600-5d34-4109-8f07-3b7fa97658a4-host-var-lib-kubelet\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569522 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-socket-dir\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569551 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bae269a1-2dfb-42d8-85ef-47939480e7ff-sys-fs\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569583 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-systemd\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569596 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-run-openvswitch\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569638 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-sysconfig\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569645 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-run\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569796 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569838 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9a30523-229a-47bb-8e6e-c7f2b20fd301-cnibin\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.569975 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-tmp\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.570007 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-ovnkube-script-lib\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.570023 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b29aaeeb-cf84-4293-b329-32de1e621742-host-run-netns\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.570070 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66832598-bf55-4875-9c98-0b96b2582b7c-iptables-alerter-script\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.571503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.570147 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b29aaeeb-cf84-4293-b329-32de1e621742-ovnkube-config\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.571998 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.570203 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-etc-tuned\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.571998 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.570376 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b29aaeeb-cf84-4293-b329-32de1e621742-ovn-node-metrics-cert\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.574998 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.574978 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmt8k\" (UniqueName: \"kubernetes.io/projected/02394e61-3140-406c-afa3-84ffc266a641-kube-api-access-rmt8k\") pod \"node-ca-bgsdc\" (UID: \"02394e61-3140-406c-afa3-84ffc266a641\") " pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.575170 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.575151 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vrq\" (UniqueName: \"kubernetes.io/projected/b29aaeeb-cf84-4293-b329-32de1e621742-kube-api-access-z5vrq\") pod \"ovnkube-node-hfwh8\" (UID: \"b29aaeeb-cf84-4293-b329-32de1e621742\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.576232 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.576212 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx7dq\" (UniqueName: \"kubernetes.io/projected/bae269a1-2dfb-42d8-85ef-47939480e7ff-kube-api-access-qx7dq\") pod \"aws-ebs-csi-driver-node-5w2kc\" (UID: \"bae269a1-2dfb-42d8-85ef-47939480e7ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.577547 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.577525 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 17:08:34.577640 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.577551 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 17:08:34.577640 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.577567 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tbq8l for pod openshift-network-diagnostics/network-check-target-vmndt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:34.577755 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.577641 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l podName:4cc3bdcb-b505-48e8-9794-63d6117a1163 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:35.077623802 +0000 UTC m=+23.150525136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tbq8l" (UniqueName: "kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l") pod "network-check-target-vmndt" (UID: "4cc3bdcb-b505-48e8-9794-63d6117a1163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:34.577755 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.577746 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nzfj\" (UniqueName: \"kubernetes.io/projected/c9a30523-229a-47bb-8e6e-c7f2b20fd301-kube-api-access-9nzfj\") pod \"multus-additional-cni-plugins-dz454\" (UID: \"c9a30523-229a-47bb-8e6e-c7f2b20fd301\") " pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.577876 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.577787 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xchs9\" (UniqueName: \"kubernetes.io/projected/66832598-bf55-4875-9c98-0b96b2582b7c-kube-api-access-xchs9\") pod \"iptables-alerter-ps5c8\" (UID: \"66832598-bf55-4875-9c98-0b96b2582b7c\") " pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.578832 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.578814 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snvhp\" (UniqueName: \"kubernetes.io/projected/96f76600-5d34-4109-8f07-3b7fa97658a4-kube-api-access-snvhp\") pod \"multus-zgbmj\" (UID: \"96f76600-5d34-4109-8f07-3b7fa97658a4\") " pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.579674 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.579657 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfxrp\" (UniqueName: \"kubernetes.io/projected/6e09d3ea-b5e9-4f78-be5d-c9ec9258d497-kube-api-access-gfxrp\") pod \"tuned-nxwnb\" (UID: \"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497\") " pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.580383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.580365 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q7h4\" (UniqueName: \"kubernetes.io/projected/36380f08-07a9-4f59-9a84-8a639a5943cd-kube-api-access-7q7h4\") pod \"node-resolver-r97tb\" (UID: \"36380f08-07a9-4f59-9a84-8a639a5943cd\") " pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.581153 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.581138 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6jxx\" (UniqueName: \"kubernetes.io/projected/d590b05d-a3ad-434d-bc55-eb1f23c0e214-kube-api-access-l6jxx\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:34.657186 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.657095 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:34.661955 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.661936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r97tb" Sep 07 17:08:34.665605 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.665582 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e168995_8f48_4329_b17f_e1f39943efc8.slice/crio-f84ff45d4b9640afd7019f3d66abf91967200925877726f0895f7e702f88c9d0 WatchSource:0}: Error finding container f84ff45d4b9640afd7019f3d66abf91967200925877726f0895f7e702f88c9d0: Status 404 returned error can't find the container with id f84ff45d4b9640afd7019f3d66abf91967200925877726f0895f7e702f88c9d0 Sep 07 17:08:34.668846 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.668822 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bgsdc" Sep 07 17:08:34.672074 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.672048 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36380f08_07a9_4f59_9a84_8a639a5943cd.slice/crio-ed847e055a0afbae753777596b683c45b32b24cac8f03d7c38e48f049e0351b0 WatchSource:0}: Error finding container ed847e055a0afbae753777596b683c45b32b24cac8f03d7c38e48f049e0351b0: Status 404 returned error can't find the container with id ed847e055a0afbae753777596b683c45b32b24cac8f03d7c38e48f049e0351b0 Sep 07 17:08:34.673210 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.673186 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zgbmj" Sep 07 17:08:34.677134 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.677112 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02394e61_3140_406c_afa3_84ffc266a641.slice/crio-6ecfb0fb324c6d945ce53c7312a58cf81cbaeb79ecfd33dfa7a30352140b57a0 WatchSource:0}: Error finding container 6ecfb0fb324c6d945ce53c7312a58cf81cbaeb79ecfd33dfa7a30352140b57a0: Status 404 returned error can't find the container with id 6ecfb0fb324c6d945ce53c7312a58cf81cbaeb79ecfd33dfa7a30352140b57a0 Sep 07 17:08:34.680181 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.680162 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dz454" Sep 07 17:08:34.681581 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.681559 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f76600_5d34_4109_8f07_3b7fa97658a4.slice/crio-336eaceccc2476fb852311c69a80404f5d7e503adf5e40f86c9b3e2b83a19751 WatchSource:0}: Error finding container 336eaceccc2476fb852311c69a80404f5d7e503adf5e40f86c9b3e2b83a19751: Status 404 returned error can't find the container with id 336eaceccc2476fb852311c69a80404f5d7e503adf5e40f86c9b3e2b83a19751 Sep 07 17:08:34.685866 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.685833 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" Sep 07 17:08:34.689446 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.689429 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" Sep 07 17:08:34.691834 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.691810 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a30523_229a_47bb_8e6e_c7f2b20fd301.slice/crio-6c1aa89c72da364477690b0f487fe67f53e16a5fa81121edd2d91da1ce66fd42 WatchSource:0}: Error finding container 6c1aa89c72da364477690b0f487fe67f53e16a5fa81121edd2d91da1ce66fd42: Status 404 returned error can't find the container with id 6c1aa89c72da364477690b0f487fe67f53e16a5fa81121edd2d91da1ce66fd42 Sep 07 17:08:34.693855 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.693808 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae269a1_2dfb_42d8_85ef_47939480e7ff.slice/crio-533868cad4792f415abe01676b3e53441b33fab0b63645704d5c6b90d0a0995a WatchSource:0}: Error finding container 533868cad4792f415abe01676b3e53441b33fab0b63645704d5c6b90d0a0995a: Status 404 returned error can't find the container with id 533868cad4792f415abe01676b3e53441b33fab0b63645704d5c6b90d0a0995a Sep 07 17:08:34.694714 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.694695 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ps5c8" Sep 07 17:08:34.699607 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.699589 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:34.699932 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.699914 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e09d3ea_b5e9_4f78_be5d_c9ec9258d497.slice/crio-dc14addec18ace3d3d211e324a38d6319d30c3ace85843ad8a8daa8ac728fcee WatchSource:0}: Error finding container dc14addec18ace3d3d211e324a38d6319d30c3ace85843ad8a8daa8ac728fcee: Status 404 returned error can't find the container with id dc14addec18ace3d3d211e324a38d6319d30c3ace85843ad8a8daa8ac728fcee Sep 07 17:08:34.705994 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.705970 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66832598_bf55_4875_9c98_0b96b2582b7c.slice/crio-b09693151f766c4c357b7083f93ea887e4626e05322364ec31250b4152cb0921 WatchSource:0}: Error finding container b09693151f766c4c357b7083f93ea887e4626e05322364ec31250b4152cb0921: Status 404 returned error can't find the container with id b09693151f766c4c357b7083f93ea887e4626e05322364ec31250b4152cb0921 Sep 07 17:08:34.708936 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:08:34.708905 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb29aaeeb_cf84_4293_b329_32de1e621742.slice/crio-1ae2d17d30e6782baec3bbd04508def83f0e89675563c735be2b93c14eb237da WatchSource:0}: Error finding container 1ae2d17d30e6782baec3bbd04508def83f0e89675563c735be2b93c14eb237da: Status 404 returned error can't find the container with id 1ae2d17d30e6782baec3bbd04508def83f0e89675563c735be2b93c14eb237da Sep 07 17:08:34.970346 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:34.970265 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:34.970471 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.970392 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:34.970471 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:34.970450 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret podName:d40dd6a8-670a-447b-9d18-7ec8645d13ef nodeName:}" failed. No retries permitted until 2026-09-07 17:08:35.970432146 +0000 UTC m=+24.043333458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret") pod "global-pull-secret-syncer-nmmvg" (UID: "d40dd6a8-670a-447b-9d18-7ec8645d13ef") : object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:35.070643 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.070599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:35.070849 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.070833 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:35.070914 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.070901 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs podName:d590b05d-a3ad-434d-bc55-eb1f23c0e214 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:36.070881969 +0000 UTC m=+24.143783294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs") pod "network-metrics-daemon-cx8hf" (UID: "d590b05d-a3ad-434d-bc55-eb1f23c0e214") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:35.171775 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.171736 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:35.171968 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.171948 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 17:08:35.172028 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.171976 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 17:08:35.172028 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.171989 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tbq8l for pod openshift-network-diagnostics/network-check-target-vmndt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:35.172155 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.172048 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l podName:4cc3bdcb-b505-48e8-9794-63d6117a1163 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:36.172030272 +0000 UTC m=+24.244931606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbq8l" (UniqueName: "kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l") pod "network-check-target-vmndt" (UID: "4cc3bdcb-b505-48e8-9794-63d6117a1163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:35.404469 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.404369 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 17:03:34 +0000 UTC" deadline="2028-05-30 02:10:36.250641256 +0000 UTC" Sep 07 17:08:35.404469 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.404410 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15129h2m0.846235952s" Sep 07 17:08:35.474794 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.474756 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r97tb" event={"ID":"36380f08-07a9-4f59-9a84-8a639a5943cd","Type":"ContainerStarted","Data":"ed847e055a0afbae753777596b683c45b32b24cac8f03d7c38e48f049e0351b0"} Sep 07 17:08:35.478401 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.478371 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6ngms" event={"ID":"9e168995-8f48-4329-b17f-e1f39943efc8","Type":"ContainerStarted","Data":"f84ff45d4b9640afd7019f3d66abf91967200925877726f0895f7e702f88c9d0"} Sep 07 17:08:35.489588 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.489464 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal_778133e8e3dd073d85d9bb9b749e3d88/setup/0.log" Sep 07 17:08:35.489588 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.489504 2563 generic.go:356] "Generic (PLEG): container finished" podID="778133e8e3dd073d85d9bb9b749e3d88" containerID="34ef9cd6f7d635b6a3b836f7c3822a27ed03e727d438a128e0dbd32bae9ec3dd" exitCode=0 Sep 07 17:08:35.489588 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.489575 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" event={"ID":"778133e8e3dd073d85d9bb9b749e3d88","Type":"ContainerDied","Data":"34ef9cd6f7d635b6a3b836f7c3822a27ed03e727d438a128e0dbd32bae9ec3dd"} Sep 07 17:08:35.489857 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.489622 2563 scope.go:117] "RemoveContainer" containerID="1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d" Sep 07 17:08:35.490102 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.490082 2563 scope.go:117] "RemoveContainer" containerID="1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d" Sep 07 17:08:35.503109 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.503027 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ps5c8" event={"ID":"66832598-bf55-4875-9c98-0b96b2582b7c","Type":"ContainerStarted","Data":"b09693151f766c4c357b7083f93ea887e4626e05322364ec31250b4152cb0921"} Sep 07 17:08:35.509002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.508927 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" event={"ID":"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497","Type":"ContainerStarted","Data":"dc14addec18ace3d3d211e324a38d6319d30c3ace85843ad8a8daa8ac728fcee"} Sep 07 17:08:35.511004 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.510974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerStarted","Data":"6c1aa89c72da364477690b0f487fe67f53e16a5fa81121edd2d91da1ce66fd42"} Sep 07 17:08:35.516256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.516228 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zgbmj" event={"ID":"96f76600-5d34-4109-8f07-3b7fa97658a4","Type":"ContainerStarted","Data":"336eaceccc2476fb852311c69a80404f5d7e503adf5e40f86c9b3e2b83a19751"} Sep 07 17:08:35.524877 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.524837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bgsdc" event={"ID":"02394e61-3140-406c-afa3-84ffc266a641","Type":"ContainerStarted","Data":"6ecfb0fb324c6d945ce53c7312a58cf81cbaeb79ecfd33dfa7a30352140b57a0"} Sep 07 17:08:35.527782 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.527722 2563 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal_openshift-machine-config-operator_778133e8e3dd073d85d9bb9b749e3d88_0 in pod sandbox bc768d841b9383a49de7c06f22b3377a627c05c65bfb8de469ebd1d34200ead4 from index: no such id: '1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d'" containerID="1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d" Sep 07 17:08:35.527891 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.527811 2563 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal_openshift-machine-config-operator_778133e8e3dd073d85d9bb9b749e3d88_0 in pod sandbox bc768d841b9383a49de7c06f22b3377a627c05c65bfb8de469ebd1d34200ead4 from index: no such id: '1227a2db078241ac1f4dbfef50bacae294292e4cf48e7ee49fe4153f6a707d7d'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal_openshift-machine-config-operator(778133e8e3dd073d85d9bb9b749e3d88)\"" logger="UnhandledError" Sep 07 17:08:35.530100 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.530073 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"1ae2d17d30e6782baec3bbd04508def83f0e89675563c735be2b93c14eb237da"} Sep 07 17:08:35.535564 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.535425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" event={"ID":"bae269a1-2dfb-42d8-85ef-47939480e7ff","Type":"ContainerStarted","Data":"533868cad4792f415abe01676b3e53441b33fab0b63645704d5c6b90d0a0995a"} Sep 07 17:08:35.979160 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:35.979077 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:35.979334 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.979251 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:35.979334 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:35.979313 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret podName:d40dd6a8-670a-447b-9d18-7ec8645d13ef nodeName:}" failed. No retries permitted until 2026-09-07 17:08:37.9792945 +0000 UTC m=+26.052195814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret") pod "global-pull-secret-syncer-nmmvg" (UID: "d40dd6a8-670a-447b-9d18-7ec8645d13ef") : object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:36.083468 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.083423 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:36.083648 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.083606 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:36.083709 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.083666 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs podName:d590b05d-a3ad-434d-bc55-eb1f23c0e214 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:38.083647735 +0000 UTC m=+26.156549066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs") pod "network-metrics-daemon-cx8hf" (UID: "d590b05d-a3ad-434d-bc55-eb1f23c0e214") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:36.184703 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.184670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:36.184909 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.184890 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 17:08:36.184970 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.184912 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 17:08:36.184970 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.184925 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tbq8l for pod openshift-network-diagnostics/network-check-target-vmndt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:36.185060 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.184980 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l podName:4cc3bdcb-b505-48e8-9794-63d6117a1163 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:38.184962353 +0000 UTC m=+26.257863680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbq8l" (UniqueName: "kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l") pod "network-check-target-vmndt" (UID: "4cc3bdcb-b505-48e8-9794-63d6117a1163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:36.405376 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.405330 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 17:03:34 +0000 UTC" deadline="2028-05-13 15:35:41.214939432 +0000 UTC" Sep 07 17:08:36.405376 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.405367 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14734h27m4.809576999s" Sep 07 17:08:36.425943 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.425894 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:36.426136 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.426091 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:36.426648 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.426599 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:36.426851 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.426835 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:36.426990 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.426942 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:36.427065 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:36.426720 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:36.554338 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.554299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" event={"ID":"778133e8e3dd073d85d9bb9b749e3d88","Type":"ContainerStarted","Data":"f264bc25b59ce2067b6e39548ea595e25b463dcb2d242f34e2a6542dc850de70"} Sep 07 17:08:36.568381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:36.567287 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-85.ec2.internal" podStartSLOduration=3.567268447 podStartE2EDuration="3.567268447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:36.567018318 +0000 UTC m=+24.639919653" watchObservedRunningTime="2026-09-07 17:08:36.567268447 +0000 UTC m=+24.640169781" Sep 07 17:08:37.999235 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:37.998620 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:37.999235 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:37.998829 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:37.999235 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:37.998896 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret podName:d40dd6a8-670a-447b-9d18-7ec8645d13ef nodeName:}" failed. No retries permitted until 2026-09-07 17:08:41.998878225 +0000 UTC m=+30.071779540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret") pod "global-pull-secret-syncer-nmmvg" (UID: "d40dd6a8-670a-447b-9d18-7ec8645d13ef") : object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:38.099599 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:38.099559 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:38.099861 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.099709 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:38.099861 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.099793 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs podName:d590b05d-a3ad-434d-bc55-eb1f23c0e214 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:42.099774836 +0000 UTC m=+30.172676154 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs") pod "network-metrics-daemon-cx8hf" (UID: "d590b05d-a3ad-434d-bc55-eb1f23c0e214") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:38.200862 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:38.200825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:38.201047 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.201012 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 17:08:38.201047 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.201033 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 17:08:38.201047 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.201047 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tbq8l for pod openshift-network-diagnostics/network-check-target-vmndt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:38.201205 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.201108 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l podName:4cc3bdcb-b505-48e8-9794-63d6117a1163 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:42.201090084 +0000 UTC m=+30.273991406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbq8l" (UniqueName: "kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l") pod "network-check-target-vmndt" (UID: "4cc3bdcb-b505-48e8-9794-63d6117a1163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:38.421585 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:38.421495 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:38.421765 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.421633 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:38.422053 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:38.422033 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:38.422159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.422139 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:38.422273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:38.422258 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:38.422362 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:38.422340 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:40.421057 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:40.421019 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:40.421508 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:40.421169 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:40.421628 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:40.421608 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:40.421820 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:40.421766 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:40.422134 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:40.422002 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:40.422134 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:40.422100 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:42.033920 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:42.033568 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:42.033920 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.033848 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:42.033920 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.033929 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret podName:d40dd6a8-670a-447b-9d18-7ec8645d13ef nodeName:}" failed. No retries permitted until 2026-09-07 17:08:50.033909242 +0000 UTC m=+38.106810559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret") pod "global-pull-secret-syncer-nmmvg" (UID: "d40dd6a8-670a-447b-9d18-7ec8645d13ef") : object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:42.135010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:42.134890 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:42.135215 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.135056 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:42.135215 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.135136 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs podName:d590b05d-a3ad-434d-bc55-eb1f23c0e214 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:50.135117209 +0000 UTC m=+38.208018519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs") pod "network-metrics-daemon-cx8hf" (UID: "d590b05d-a3ad-434d-bc55-eb1f23c0e214") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:42.235944 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:42.235863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:42.236139 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.236117 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 17:08:42.236192 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.236149 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 17:08:42.236192 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.236163 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tbq8l for pod openshift-network-diagnostics/network-check-target-vmndt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:42.236327 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.236239 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l podName:4cc3bdcb-b505-48e8-9794-63d6117a1163 nodeName:}" failed. No retries permitted until 2026-09-07 17:08:50.236217695 +0000 UTC m=+38.309119006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbq8l" (UniqueName: "kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l") pod "network-check-target-vmndt" (UID: "4cc3bdcb-b505-48e8-9794-63d6117a1163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:42.424099 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:42.422706 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:42.424099 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:42.423266 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:42.424099 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.423538 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:42.424099 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.423536 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:42.424099 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:42.423640 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:42.424099 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:42.423846 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:43.995568 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:43.995538 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 17:08:44.421486 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:44.421399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:44.421771 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:44.421399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:44.421771 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:44.421546 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:44.421771 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:44.421557 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:44.421930 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:44.421794 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:44.421930 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:44.421841 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:46.420940 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:46.420899 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:46.420940 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:46.420935 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:46.421401 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:46.420899 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:46.421401 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:46.421047 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:46.421401 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:46.421130 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:46.421401 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:46.421230 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:48.420866 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:48.420827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:48.420866 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:48.420865 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:48.421383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:48.420827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:48.421383 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:48.420958 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:48.421383 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:48.421009 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:48.421383 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:48.421073 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:50.099150 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:50.099115 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:50.099712 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.099304 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:50.099712 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.099368 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret podName:d40dd6a8-670a-447b-9d18-7ec8645d13ef nodeName:}" failed. No retries permitted until 2026-09-07 17:09:06.099349252 +0000 UTC m=+54.172250564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret") pod "global-pull-secret-syncer-nmmvg" (UID: "d40dd6a8-670a-447b-9d18-7ec8645d13ef") : object "kube-system"/"original-pull-secret" not registered Sep 07 17:08:50.200088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:50.200053 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:50.200262 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.200215 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:50.200327 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.200289 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs podName:d590b05d-a3ad-434d-bc55-eb1f23c0e214 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:06.200268509 +0000 UTC m=+54.273169826 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs") pod "network-metrics-daemon-cx8hf" (UID: "d590b05d-a3ad-434d-bc55-eb1f23c0e214") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:08:50.301169 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:50.301135 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:50.301346 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.301272 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 17:08:50.301346 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.301290 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 17:08:50.301346 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.301303 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tbq8l for pod openshift-network-diagnostics/network-check-target-vmndt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:50.301490 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.301362 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l podName:4cc3bdcb-b505-48e8-9794-63d6117a1163 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:06.301345382 +0000 UTC m=+54.374246715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbq8l" (UniqueName: "kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l") pod "network-check-target-vmndt" (UID: "4cc3bdcb-b505-48e8-9794-63d6117a1163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:08:50.421146 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:50.421066 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:50.421293 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:50.421065 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:50.421293 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.421202 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:50.421293 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:50.421065 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:50.421443 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.421244 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:50.421443 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:50.421360 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:52.422278 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.422029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:52.422901 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.422093 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:52.422901 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:52.422357 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:52.422901 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:52.422435 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:52.422901 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.422125 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:52.422901 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:52.422510 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:52.590368 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.590332 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9a30523-229a-47bb-8e6e-c7f2b20fd301" containerID="4ba2dad696f83e8c564c95ad4dca256c9cdda0098f84eb8be34772942a8295d6" exitCode=0 Sep 07 17:08:52.590546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.590428 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerDied","Data":"4ba2dad696f83e8c564c95ad4dca256c9cdda0098f84eb8be34772942a8295d6"} Sep 07 17:08:52.592205 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.592182 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zgbmj" event={"ID":"96f76600-5d34-4109-8f07-3b7fa97658a4","Type":"ContainerStarted","Data":"6268fecbfeb33550c091a192b1e02b822d996f5428659838a8355764d52ab6d8"} Sep 07 17:08:52.593892 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.593861 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bgsdc" event={"ID":"02394e61-3140-406c-afa3-84ffc266a641","Type":"ContainerStarted","Data":"46a295eeb8d944a8eb39958b9d5e1bdc0fbd66c40be735e4ff93b0bc47d58f58"} Sep 07 17:08:52.598830 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.598796 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"3379927b546dc6d98509cf1b111c89224d38364413435676ea29a4d70b13cf2f"} Sep 07 17:08:52.598934 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.598873 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"66649b8f31af1d8e7c2144b6faf164b795dc6c91d1826e7d57847e95d1089c02"} Sep 07 17:08:52.598934 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.598922 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"ad11ab5b23719140c45141e81b95fbf34b230e2e265d7f2fdc97d51512832e38"} Sep 07 17:08:52.599042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.598964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"ae811194c48ed3da9154483891f5e2698c4955a8e081fbb0a7425b57ff94d776"} Sep 07 17:08:52.599042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.598992 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"11d4bfe9ff7812d175aa29d83852af0d26adc011b851db5319a6a5410b915af5"} Sep 07 17:08:52.599042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.599025 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"97e5ed1e80326666ef21196cabde52cf090bf181f218a09d1b56e2a695d337c1"} Sep 07 17:08:52.600396 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.600378 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" event={"ID":"bae269a1-2dfb-42d8-85ef-47939480e7ff","Type":"ContainerStarted","Data":"efab15d561f3c8496714b721dfc301dcef542c587ec62610aee08ad3d9def0c5"} Sep 07 17:08:52.601977 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.601948 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r97tb" event={"ID":"36380f08-07a9-4f59-9a84-8a639a5943cd","Type":"ContainerStarted","Data":"b5fb56714be8fc022384e4637516b1c59945fee505429fb975afebd400c0aae2"} Sep 07 17:08:52.603696 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.603678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6ngms" event={"ID":"9e168995-8f48-4329-b17f-e1f39943efc8","Type":"ContainerStarted","Data":"37ed31bfc9e563558b540f61414424044f8cae12531a0855a621d763bf87bb9a"} Sep 07 17:08:52.605292 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.605270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" event={"ID":"6e09d3ea-b5e9-4f78-be5d-c9ec9258d497","Type":"ContainerStarted","Data":"72d14b69575619331f9077a289dfd07b387824f3df3c392f1b23813620e43c37"} Sep 07 17:08:52.647139 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.647079 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nxwnb" podStartSLOduration=3.6410108770000003 podStartE2EDuration="20.647056566s" totalImagesPullingTime="17.006045689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:52.647003339 +0000 UTC m=+40.719904672" watchObservedRunningTime="2026-09-07 17:08:52.647056566 +0000 UTC m=+40.719957900" Sep 07 17:08:52.665922 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.665865 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zgbmj" podStartSLOduration=3.465843726 podStartE2EDuration="20.665849889s" totalImagesPullingTime="17.200006163s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:52.665450325 +0000 UTC m=+40.738351658" watchObservedRunningTime="2026-09-07 17:08:52.665849889 +0000 UTC m=+40.738751221" Sep 07 17:08:52.719051 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.718990 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6ngms" podStartSLOduration=7.4292456 podStartE2EDuration="20.718970479s" totalImagesPullingTime="13.289724879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:52.687614844 +0000 UTC m=+40.760516171" watchObservedRunningTime="2026-09-07 17:08:52.718970479 +0000 UTC m=+40.791871813" Sep 07 17:08:52.737632 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.737595 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r97tb" podStartSLOduration=3.740319886 podStartE2EDuration="20.737584179s" totalImagesPullingTime="16.997264293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:52.718697241 +0000 UTC m=+40.791598575" watchObservedRunningTime="2026-09-07 17:08:52.737584179 +0000 UTC m=+40.810485560" Sep 07 17:08:52.737721 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.737660 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bgsdc" podStartSLOduration=3.721442529 podStartE2EDuration="20.737656041s" totalImagesPullingTime="17.016213512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:52.737210333 +0000 UTC m=+40.810111666" watchObservedRunningTime="2026-09-07 17:08:52.737656041 +0000 UTC m=+40.810557373" Sep 07 17:08:52.814840 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:52.814820 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 07 17:08:53.451875 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:53.451700 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T17:08:52.814836383Z","UUID":"5e1fe746-31aa-4e10-9422-9d30956e4712","Handler":null,"Name":"","Endpoint":""} Sep 07 17:08:53.453444 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:53.453421 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 07 17:08:53.453570 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:53.453456 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 07 17:08:53.613643 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:53.613521 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ps5c8" event={"ID":"66832598-bf55-4875-9c98-0b96b2582b7c","Type":"ContainerStarted","Data":"f7944ec73145f76d23d38b9b0b0e8aa54273c0f53b6f2342b840a16b081106eb"} Sep 07 17:08:53.616634 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:53.616604 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" event={"ID":"bae269a1-2dfb-42d8-85ef-47939480e7ff","Type":"ContainerStarted","Data":"cf14e43bb0154ecd03d924524f70474ddda4dfd58dc76d833c4c9a35284a48b1"} Sep 07 17:08:53.628842 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:53.628792 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ps5c8" podStartSLOduration=4.618332852 podStartE2EDuration="21.62877333s" totalImagesPullingTime="17.010440478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:53.628312129 +0000 UTC m=+41.701213462" watchObservedRunningTime="2026-09-07 17:08:53.62877333 +0000 UTC m=+41.701674665" Sep 07 17:08:54.421412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.421381 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:54.421584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.421424 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:54.421584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.421384 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:54.421674 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:54.421611 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:54.421874 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:54.421845 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:54.422019 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:54.421986 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:54.622532 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.622454 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"bb24edf825f838171e435b8988864da50a7284ca8fd03ba89ea58900f47ea3b0"} Sep 07 17:08:54.625062 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.625006 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" event={"ID":"bae269a1-2dfb-42d8-85ef-47939480e7ff","Type":"ContainerStarted","Data":"8d69b8ee01d1023bc8b183f80a6136aed42222e3e1e901051bdf9de6aa59067d"} Sep 07 17:08:54.642678 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.642593 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5w2kc" podStartSLOduration=4.119208971 podStartE2EDuration="22.64257588s" totalImagesPullingTime="18.523366909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:54.642236088 +0000 UTC m=+42.715137422" watchObservedRunningTime="2026-09-07 17:08:54.64257588 +0000 UTC m=+42.715477213" Sep 07 17:08:54.657599 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.657569 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:54.658206 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:54.658185 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:56.420802 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:56.420770 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:56.421244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:56.420770 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:56.421244 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:56.420868 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:56.421244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:56.420884 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:56.421244 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:56.421021 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:56.421244 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:56.421116 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:56.634457 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:56.634292 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" event={"ID":"b29aaeeb-cf84-4293-b329-32de1e621742","Type":"ContainerStarted","Data":"6f196651d91603be640fb300c21fac30cabe2c7c140cb5ff6680a092e3e27b32"} Sep 07 17:08:56.646428 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:56.646306 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:56.662634 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:56.662594 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:56.679774 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:56.679508 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" podStartSLOduration=7.645643306 podStartE2EDuration="24.679488466s" totalImagesPullingTime="17.03384516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:08:56.678644526 +0000 UTC m=+44.751545875" watchObservedRunningTime="2026-09-07 17:08:56.679488466 +0000 UTC m=+44.752389793" Sep 07 17:08:57.026130 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.026091 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:57.026275 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.026225 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 17:08:57.026791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.026774 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6ngms" Sep 07 17:08:57.638440 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.638402 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9a30523-229a-47bb-8e6e-c7f2b20fd301" containerID="868103ec68beabd3abbd634af126666e2c26ed692dae1b58d9e1795e90079ecf" exitCode=0 Sep 07 17:08:57.638900 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.638488 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerDied","Data":"868103ec68beabd3abbd634af126666e2c26ed692dae1b58d9e1795e90079ecf"} Sep 07 17:08:57.638900 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.638554 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 17:08:57.639011 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.638993 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:57.653977 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:57.653955 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:08:58.420836 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.420796 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:58.420836 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.420820 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:58.420836 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.420836 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:58.421240 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:58.420943 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:58.421240 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:58.421091 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:58.421240 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:58.421190 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:58.428048 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.428021 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nmmvg"] Sep 07 17:08:58.430511 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.430484 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vmndt"] Sep 07 17:08:58.433086 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.433065 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cx8hf"] Sep 07 17:08:58.641219 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.641189 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:08:58.641917 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.641225 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:08:58.641917 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.641359 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 17:08:58.641917 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:58.641379 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:08:58.641917 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:58.641390 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:08:58.641917 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:58.641779 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:08:58.641917 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:08:58.641863 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:08:59.644922 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:59.644713 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9a30523-229a-47bb-8e6e-c7f2b20fd301" containerID="0f42086686dd472323d8e6125ae461db8632883b2e67f25a3823b11c3f910a47" exitCode=0 Sep 07 17:08:59.645416 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:59.644765 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerDied","Data":"0f42086686dd472323d8e6125ae461db8632883b2e67f25a3823b11c3f910a47"} Sep 07 17:08:59.645416 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:08:59.645104 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 17:09:00.421318 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:00.421286 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:00.421318 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:00.421314 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:00.421548 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:00.421286 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:00.421548 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:00.421408 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:09:00.421548 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:00.421481 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:09:00.421657 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:00.421562 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:09:00.949539 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:00.949512 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:09:01.651763 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:01.651662 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9a30523-229a-47bb-8e6e-c7f2b20fd301" containerID="3641b3840bda6633d774d1eb025a26b5a78f8e0caf85ff9573b399f54c7e1263" exitCode=0 Sep 07 17:09:01.651763 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:01.651721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerDied","Data":"3641b3840bda6633d774d1eb025a26b5a78f8e0caf85ff9573b399f54c7e1263"} Sep 07 17:09:02.421747 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:02.421644 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:02.422175 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:02.421759 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:02.422175 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:02.421804 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:02.422175 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:02.421806 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:09:02.422175 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:02.421882 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:09:02.422175 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:02.421969 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:09:04.420767 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.420724 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:04.421195 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.420742 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:04.421195 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:04.420878 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cx8hf" podUID="d590b05d-a3ad-434d-bc55-eb1f23c0e214" Sep 07 17:09:04.421195 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:04.420962 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nmmvg" podUID="d40dd6a8-670a-447b-9d18-7ec8645d13ef" Sep 07 17:09:04.421195 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.420736 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:04.421195 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:04.421063 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmndt" podUID="4cc3bdcb-b505-48e8-9794-63d6117a1163" Sep 07 17:09:04.720516 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.720430 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-85.ec2.internal" event="NodeReady" Sep 07 17:09:04.720677 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.720571 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 07 17:09:04.760814 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.760778 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cfdfbdc99-nltls"] Sep 07 17:09:04.784342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.784129 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2tgv4"] Sep 07 17:09:04.784342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.784278 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.787010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.786985 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 07 17:09:04.787162 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.787034 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cxf5h\"" Sep 07 17:09:04.787275 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.787255 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 07 17:09:04.787275 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.787265 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 07 17:09:04.794583 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.794559 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 07 17:09:04.798001 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.797976 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2l6hx"] Sep 07 17:09:04.798149 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.798134 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:04.799741 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.799692 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 07 17:09:04.799932 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.799899 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq2rt\"" Sep 07 17:09:04.800013 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.799935 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 07 17:09:04.812043 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.812018 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfdfbdc99-nltls"] Sep 07 17:09:04.812167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.812057 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2tgv4"] Sep 07 17:09:04.812167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.812081 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2l6hx"] Sep 07 17:09:04.812167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.812139 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:04.814425 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.814378 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 07 17:09:04.814528 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.814462 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5zx9s\"" Sep 07 17:09:04.814576 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.814408 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 07 17:09:04.814685 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.814669 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 07 17:09:04.904668 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904633 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.904855 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904682 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntfs8\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-kube-api-access-ntfs8\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.904855 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-registry-certificates\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.904855 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:04.904855 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904844 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-trusted-ca\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.905041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d2c8366-312c-4f0d-929b-ad4245f7c236-config-volume\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:04.905041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lfqq\" (UniqueName: \"kubernetes.io/projected/d3b8ac2d-79b3-4163-898f-aa94677f513b-kube-api-access-7lfqq\") pod \"ingress-canary-2l6hx\" (UID: \"d3b8ac2d-79b3-4163-898f-aa94677f513b\") " pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:04.905041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-image-registry-private-configuration\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.905041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.904958 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-installation-pull-secrets\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.905041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.905007 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bwmx\" (UniqueName: \"kubernetes.io/projected/2d2c8366-312c-4f0d-929b-ad4245f7c236-kube-api-access-8bwmx\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:04.905041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.905036 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d2c8366-312c-4f0d-929b-ad4245f7c236-tmp-dir\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:04.905266 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.905051 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3b8ac2d-79b3-4163-898f-aa94677f513b-cert\") pod \"ingress-canary-2l6hx\" (UID: \"d3b8ac2d-79b3-4163-898f-aa94677f513b\") " pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:04.905266 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.905089 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5945742-663c-47cb-a119-ef0423cd37f0-ca-trust-extracted\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:04.905266 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:04.905114 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-bound-sa-token\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.005938 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.005901 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d2c8366-312c-4f0d-929b-ad4245f7c236-config-volume\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.005938 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.005941 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lfqq\" (UniqueName: \"kubernetes.io/projected/d3b8ac2d-79b3-4163-898f-aa94677f513b-kube-api-access-7lfqq\") pod \"ingress-canary-2l6hx\" (UID: \"d3b8ac2d-79b3-4163-898f-aa94677f513b\") " pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.005978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-image-registry-private-configuration\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-installation-pull-secrets\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006046 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bwmx\" (UniqueName: \"kubernetes.io/projected/2d2c8366-312c-4f0d-929b-ad4245f7c236-kube-api-access-8bwmx\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d2c8366-312c-4f0d-929b-ad4245f7c236-tmp-dir\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006094 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3b8ac2d-79b3-4163-898f-aa94677f513b-cert\") pod \"ingress-canary-2l6hx\" (UID: \"d3b8ac2d-79b3-4163-898f-aa94677f513b\") " pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5945742-663c-47cb-a119-ef0423cd37f0-ca-trust-extracted\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006157 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-bound-sa-token\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfs8\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-kube-api-access-ntfs8\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-registry-certificates\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.006639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-trusted-ca\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.006639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006551 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d2c8366-312c-4f0d-929b-ad4245f7c236-config-volume\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.006639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.006630 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5945742-663c-47cb-a119-ef0423cd37f0-ca-trust-extracted\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.007219 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.006949 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 17:09:05.007219 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.006973 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfdfbdc99-nltls: secret "image-registry-tls" not found Sep 07 17:09:05.007219 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.007019 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 17:09:05.007219 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.007032 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls podName:f5945742-663c-47cb-a119-ef0423cd37f0 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:05.507014673 +0000 UTC m=+53.579915983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls") pod "image-registry-6cfdfbdc99-nltls" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0") : secret "image-registry-tls" not found Sep 07 17:09:05.007219 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.007071 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls podName:2d2c8366-312c-4f0d-929b-ad4245f7c236 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:05.507054966 +0000 UTC m=+53.579956290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls") pod "dns-default-2tgv4" (UID: "2d2c8366-312c-4f0d-929b-ad4245f7c236") : secret "dns-default-metrics-tls" not found Sep 07 17:09:05.007493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.007294 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d2c8366-312c-4f0d-929b-ad4245f7c236-tmp-dir\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.008402 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.008377 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-registry-certificates\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.008847 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.008825 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-trusted-ca\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.011541 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.011102 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-installation-pull-secrets\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.011541 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.011125 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3b8ac2d-79b3-4163-898f-aa94677f513b-cert\") pod \"ingress-canary-2l6hx\" (UID: \"d3b8ac2d-79b3-4163-898f-aa94677f513b\") " pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:05.014291 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.014271 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lfqq\" (UniqueName: \"kubernetes.io/projected/d3b8ac2d-79b3-4163-898f-aa94677f513b-kube-api-access-7lfqq\") pod \"ingress-canary-2l6hx\" (UID: \"d3b8ac2d-79b3-4163-898f-aa94677f513b\") " pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:05.015494 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.015433 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bwmx\" (UniqueName: \"kubernetes.io/projected/2d2c8366-312c-4f0d-929b-ad4245f7c236-kube-api-access-8bwmx\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.015597 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.015554 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntfs8\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-kube-api-access-ntfs8\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.015775 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.015716 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-image-registry-private-configuration\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.015847 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.015821 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-bound-sa-token\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.124021 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.123986 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2l6hx" Sep 07 17:09:05.511089 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.510868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:05.511363 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.511098 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 17:09:05.511363 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.511121 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfdfbdc99-nltls: secret "image-registry-tls" not found Sep 07 17:09:05.511363 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.511132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:05.511363 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.511182 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls podName:f5945742-663c-47cb-a119-ef0423cd37f0 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:06.51116278 +0000 UTC m=+54.584064094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls") pod "image-registry-6cfdfbdc99-nltls" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0") : secret "image-registry-tls" not found Sep 07 17:09:05.511363 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.511243 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 17:09:05.511363 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:05.511308 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls podName:2d2c8366-312c-4f0d-929b-ad4245f7c236 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:06.511291291 +0000 UTC m=+54.584192624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls") pod "dns-default-2tgv4" (UID: "2d2c8366-312c-4f0d-929b-ad4245f7c236") : secret "dns-default-metrics-tls" not found Sep 07 17:09:05.664334 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.664308 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerStarted","Data":"dce8f1bb83f831b4facf02c8bee413e47179e1986e34198ef734e74edf01e08f"} Sep 07 17:09:05.709301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:05.709216 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2l6hx"] Sep 07 17:09:05.721742 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:05.721694 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b8ac2d_79b3_4163_898f_aa94677f513b.slice/crio-df211223cae8e423879bcdc652f3e35d200753285a67eeb8241964c4e5ee6c83 WatchSource:0}: Error finding container df211223cae8e423879bcdc652f3e35d200753285a67eeb8241964c4e5ee6c83: Status 404 returned error can't find the container with id df211223cae8e423879bcdc652f3e35d200753285a67eeb8241964c4e5ee6c83 Sep 07 17:09:06.116436 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.116400 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:06.116576 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.116551 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 17:09:06.116633 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.116616 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret podName:d40dd6a8-670a-447b-9d18-7ec8645d13ef nodeName:}" failed. No retries permitted until 2026-09-07 17:09:38.116601394 +0000 UTC m=+86.189502705 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret") pod "global-pull-secret-syncer-nmmvg" (UID: "d40dd6a8-670a-447b-9d18-7ec8645d13ef") : object "kube-system"/"original-pull-secret" not registered Sep 07 17:09:06.217059 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.217019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:06.217235 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.217160 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:09:06.217235 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.217224 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs podName:d590b05d-a3ad-434d-bc55-eb1f23c0e214 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:38.217209333 +0000 UTC m=+86.290110666 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs") pod "network-metrics-daemon-cx8hf" (UID: "d590b05d-a3ad-434d-bc55-eb1f23c0e214") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 17:09:06.318300 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.318213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:06.318470 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.318361 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 17:09:06.318470 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.318383 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 17:09:06.318470 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.318393 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tbq8l for pod openshift-network-diagnostics/network-check-target-vmndt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:09:06.318470 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.318453 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l podName:4cc3bdcb-b505-48e8-9794-63d6117a1163 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:38.318438208 +0000 UTC m=+86.391339519 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbq8l" (UniqueName: "kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l") pod "network-check-target-vmndt" (UID: "4cc3bdcb-b505-48e8-9794-63d6117a1163") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 17:09:06.421035 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.420996 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:06.421221 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.421009 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:06.421282 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.421241 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:06.423488 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.423450 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 17:09:06.423488 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.423462 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 17:09:06.423790 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.423770 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 17:09:06.423888 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.423772 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 17:09:06.423952 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.423904 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mwmqq\"" Sep 07 17:09:06.424008 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.423992 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xmgvk\"" Sep 07 17:09:06.520184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.520149 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:06.520883 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.520227 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:06.520883 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.520339 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 17:09:06.520883 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.520357 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 17:09:06.520883 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.520417 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls podName:2d2c8366-312c-4f0d-929b-ad4245f7c236 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:08.520397846 +0000 UTC m=+56.593299159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls") pod "dns-default-2tgv4" (UID: "2d2c8366-312c-4f0d-929b-ad4245f7c236") : secret "dns-default-metrics-tls" not found Sep 07 17:09:06.520883 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.520360 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfdfbdc99-nltls: secret "image-registry-tls" not found Sep 07 17:09:06.520883 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:06.520494 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls podName:f5945742-663c-47cb-a119-ef0423cd37f0 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:08.520474578 +0000 UTC m=+56.593375892 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls") pod "image-registry-6cfdfbdc99-nltls" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0") : secret "image-registry-tls" not found Sep 07 17:09:06.668092 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.668008 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2l6hx" event={"ID":"d3b8ac2d-79b3-4163-898f-aa94677f513b","Type":"ContainerStarted","Data":"df211223cae8e423879bcdc652f3e35d200753285a67eeb8241964c4e5ee6c83"} Sep 07 17:09:06.671313 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.671282 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9a30523-229a-47bb-8e6e-c7f2b20fd301" containerID="dce8f1bb83f831b4facf02c8bee413e47179e1986e34198ef734e74edf01e08f" exitCode=0 Sep 07 17:09:06.671437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:06.671322 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerDied","Data":"dce8f1bb83f831b4facf02c8bee413e47179e1986e34198ef734e74edf01e08f"} Sep 07 17:09:07.678194 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:07.678158 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9a30523-229a-47bb-8e6e-c7f2b20fd301" containerID="99da877e33b75abeb0e56167545e4b781144ba98281444bd037a2996653b6617" exitCode=0 Sep 07 17:09:07.678778 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:07.678222 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerDied","Data":"99da877e33b75abeb0e56167545e4b781144ba98281444bd037a2996653b6617"} Sep 07 17:09:08.538342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.538307 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:08.538510 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.538370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:08.538510 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:08.538493 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 17:09:08.538585 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:08.538512 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 17:09:08.538585 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:08.538518 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfdfbdc99-nltls: secret "image-registry-tls" not found Sep 07 17:09:08.538585 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:08.538577 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls podName:2d2c8366-312c-4f0d-929b-ad4245f7c236 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:12.538560886 +0000 UTC m=+60.611462216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls") pod "dns-default-2tgv4" (UID: "2d2c8366-312c-4f0d-929b-ad4245f7c236") : secret "dns-default-metrics-tls" not found Sep 07 17:09:08.538685 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:08.538594 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls podName:f5945742-663c-47cb-a119-ef0423cd37f0 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:12.538588099 +0000 UTC m=+60.611489409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls") pod "image-registry-6cfdfbdc99-nltls" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0") : secret "image-registry-tls" not found Sep 07 17:09:08.684050 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.684012 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz454" event={"ID":"c9a30523-229a-47bb-8e6e-c7f2b20fd301","Type":"ContainerStarted","Data":"a213cfe0a032325c57c6feaf4785e3ab02c0533e16b135e97407f78e5c80b203"} Sep 07 17:09:08.685428 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.685399 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2l6hx" event={"ID":"d3b8ac2d-79b3-4163-898f-aa94677f513b","Type":"ContainerStarted","Data":"30a4553349834c8728fef388ad50e238154b713181dd7c766557ca44078e21c8"} Sep 07 17:09:08.707630 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.707544 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dz454" podStartSLOduration=9.794486539 podStartE2EDuration="36.70753055s" totalImagesPullingTime="26.913044011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:08.705633377 +0000 UTC m=+56.778534709" watchObservedRunningTime="2026-09-07 17:09:08.70753055 +0000 UTC m=+56.780431874" Sep 07 17:09:08.722446 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.722396 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2l6hx" podStartSLOduration=2.443094088 podStartE2EDuration="4.722377715s" totalImagesPullingTime="2.279283627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:08.721449966 +0000 UTC m=+56.794351301" watchObservedRunningTime="2026-09-07 17:09:08.722377715 +0000 UTC m=+56.795279049" Sep 07 17:09:08.872523 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.872490 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:08.972483 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:08.972403 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:09.387400 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:09.387363 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:09.455900 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:09.455874 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:09.736301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:09.736227 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:09.806275 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:09.806248 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:10.390133 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:10.390101 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:10.451086 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:10.451051 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:10.524650 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:10.524620 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:11.176549 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:11.176519 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:11.249167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:11.249140 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:12.321652 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:12.321621 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:12.425476 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:12.425436 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:12.566854 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:12.566811 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:12.567031 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:12.566871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:12.567031 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:12.566958 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 17:09:12.567031 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:12.566979 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfdfbdc99-nltls: secret "image-registry-tls" not found Sep 07 17:09:12.567140 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:12.567028 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 17:09:12.567140 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:12.567034 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls podName:f5945742-663c-47cb-a119-ef0423cd37f0 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:20.567019738 +0000 UTC m=+68.639921053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls") pod "image-registry-6cfdfbdc99-nltls" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0") : secret "image-registry-tls" not found Sep 07 17:09:12.567140 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:12.567091 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls podName:2d2c8366-312c-4f0d-929b-ad4245f7c236 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:20.567072702 +0000 UTC m=+68.639974013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls") pod "dns-default-2tgv4" (UID: "2d2c8366-312c-4f0d-929b-ad4245f7c236") : secret "dns-default-metrics-tls" not found Sep 07 17:09:13.723645 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:13.723615 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:13.825682 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:13.825654 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:20.623496 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:20.623455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:20.624014 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:20.623505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:20.624014 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:20.623600 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 17:09:20.624014 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:20.623659 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls podName:2d2c8366-312c-4f0d-929b-ad4245f7c236 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:36.623646667 +0000 UTC m=+84.696547977 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls") pod "dns-default-2tgv4" (UID: "2d2c8366-312c-4f0d-929b-ad4245f7c236") : secret "dns-default-metrics-tls" not found Sep 07 17:09:20.624014 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:20.623600 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 17:09:20.624014 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:20.623711 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfdfbdc99-nltls: secret "image-registry-tls" not found Sep 07 17:09:20.624014 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:20.623806 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls podName:f5945742-663c-47cb-a119-ef0423cd37f0 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:36.62378758 +0000 UTC m=+84.696688892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls") pod "image-registry-6cfdfbdc99-nltls" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0") : secret "image-registry-tls" not found Sep 07 17:09:22.947147 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.947100 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hk99l"] Sep 07 17:09:22.947716 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.947688 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:22.988747 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.988705 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hk99l"] Sep 07 17:09:22.988907 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.988833 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:22.990568 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.990542 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vzbr9\"" Sep 07 17:09:22.990694 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.990611 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 07 17:09:22.990842 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.990826 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 07 17:09:22.991268 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.991251 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 07 17:09:22.991345 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:22.991329 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 07 17:09:23.013149 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.013122 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:23.144939 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.144904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d45929c7-ef4e-4f12-b694-52cb7b8eb013-data-volume\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.145098 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.144978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.145098 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.145001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xw4z\" (UniqueName: \"kubernetes.io/projected/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-api-access-6xw4z\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.145098 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.145041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d45929c7-ef4e-4f12-b694-52cb7b8eb013-crio-socket\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.145205 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.145096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.245510 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.245483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d45929c7-ef4e-4f12-b694-52cb7b8eb013-crio-socket\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.245641 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.245529 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.245641 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.245566 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d45929c7-ef4e-4f12-b694-52cb7b8eb013-data-volume\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.245641 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.245602 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d45929c7-ef4e-4f12-b694-52cb7b8eb013-crio-socket\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.245787 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.245679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.245787 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.245710 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 17:09:23.245787 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.245717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xw4z\" (UniqueName: \"kubernetes.io/projected/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-api-access-6xw4z\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.245930 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.245800 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:23.745783418 +0000 UTC m=+71.818684728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : secret "insights-runtime-extractor-tls" not found Sep 07 17:09:23.245930 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.245837 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 17:09:23.245930 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.245907 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:23.745893814 +0000 UTC m=+71.818795129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : configmap "kube-rbac-proxy" not found Sep 07 17:09:23.245930 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.245921 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d45929c7-ef4e-4f12-b694-52cb7b8eb013-data-volume\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.248558 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.248535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:23.259550 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.259528 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xw4z\" (UniqueName: \"kubernetes.io/projected/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-api-access-6xw4z\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.328177 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.328143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:23.414884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.414847 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:23.749830 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.749801 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.749994 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.749912 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 17:09:23.749994 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:23.749932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:23.749994 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.749974 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:24.749957649 +0000 UTC m=+72.822858959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : configmap "kube-rbac-proxy" not found Sep 07 17:09:23.750115 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.750021 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 17:09:23.750115 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:23.750064 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:24.750051989 +0000 UTC m=+72.822953303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : secret "insights-runtime-extractor-tls" not found Sep 07 17:09:24.756601 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:24.756563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:24.757010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:24.756654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:24.757010 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:24.756724 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 17:09:24.757010 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:24.756790 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 17:09:24.757010 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:24.756803 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:26.756786202 +0000 UTC m=+74.829687531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : secret "insights-runtime-extractor-tls" not found Sep 07 17:09:24.757010 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:24.756853 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:26.756839015 +0000 UTC m=+74.829740326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : configmap "kube-rbac-proxy" not found Sep 07 17:09:24.812175 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:24.812140 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:24.884927 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:24.884899 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:25.098239 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:25.098171 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r97tb_36380f08-07a9-4f59-9a84-8a639a5943cd/dns-node-resolver/0.log" Sep 07 17:09:25.242428 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:25.242394 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:25.299113 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:25.299082 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:25.520788 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:25.520758 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:25.620969 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:25.620936 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:25.900247 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:25.900172 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bgsdc_02394e61-3140-406c-afa3-84ffc266a641/node-ca/0.log" Sep 07 17:09:26.179887 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:26.179809 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:26.249571 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:26.249546 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:26.498278 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:26.498253 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2l6hx_d3b8ac2d-79b3-4163-898f-aa94677f513b/serve-healthcheck-canary/0.log" Sep 07 17:09:26.772989 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:26.772906 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:26.773132 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:26.772997 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:26.773132 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:26.773069 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 17:09:26.773214 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:26.773135 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:30.7731212 +0000 UTC m=+78.846022516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : secret "insights-runtime-extractor-tls" not found Sep 07 17:09:26.773214 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:26.773072 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 17:09:26.773214 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:26.773186 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:30.77317402 +0000 UTC m=+78.846075348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : configmap "kube-rbac-proxy" not found Sep 07 17:09:27.118534 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.118451 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-9r6dk"] Sep 07 17:09:27.140010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.139968 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-9r6dk"] Sep 07 17:09:27.140147 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.140099 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.142400 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.142379 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 07 17:09:27.142400 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.142393 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 07 17:09:27.142780 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.142761 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 07 17:09:27.142871 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.142764 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b98k4\"" Sep 07 17:09:27.143256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.143239 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 07 17:09:27.162575 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.162544 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:27.232410 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.232376 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:27.277637 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.277604 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1549cc9-1b91-4990-b573-645b117a4878-signing-key\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.277764 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.277644 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p29vb\" (UniqueName: \"kubernetes.io/projected/a1549cc9-1b91-4990-b573-645b117a4878-kube-api-access-p29vb\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.277828 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.277792 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1549cc9-1b91-4990-b573-645b117a4878-signing-cabundle\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.378567 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.378483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1549cc9-1b91-4990-b573-645b117a4878-signing-key\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.378567 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.378530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p29vb\" (UniqueName: \"kubernetes.io/projected/a1549cc9-1b91-4990-b573-645b117a4878-kube-api-access-p29vb\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.378782 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.378602 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1549cc9-1b91-4990-b573-645b117a4878-signing-cabundle\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.379283 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.379265 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1549cc9-1b91-4990-b573-645b117a4878-signing-cabundle\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.380951 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.380936 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1549cc9-1b91-4990-b573-645b117a4878-signing-key\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.393793 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.393767 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p29vb\" (UniqueName: \"kubernetes.io/projected/a1549cc9-1b91-4990-b573-645b117a4878-kube-api-access-p29vb\") pod \"service-ca-75d99c484-9r6dk\" (UID: \"a1549cc9-1b91-4990-b573-645b117a4878\") " pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.448689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.448659 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-9r6dk" Sep 07 17:09:27.737140 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.737056 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-9r6dk"] Sep 07 17:09:27.740969 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:27.740943 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1549cc9_1b91_4990_b573_645b117a4878.slice/crio-3cca11136453516d9399c880fa42e84261c43d57a2cba8e5fa6533f6c6d367ed WatchSource:0}: Error finding container 3cca11136453516d9399c880fa42e84261c43d57a2cba8e5fa6533f6c6d367ed: Status 404 returned error can't find the container with id 3cca11136453516d9399c880fa42e84261c43d57a2cba8e5fa6533f6c6d367ed Sep 07 17:09:27.759026 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:27.758999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:28.275079 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:28.275044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:28.735493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:28.735407 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-9r6dk" event={"ID":"a1549cc9-1b91-4990-b573-645b117a4878","Type":"ContainerStarted","Data":"3cca11136453516d9399c880fa42e84261c43d57a2cba8e5fa6533f6c6d367ed"} Sep 07 17:09:30.107372 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:30.107288 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:30.742807 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:30.742770 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-9r6dk" event={"ID":"a1549cc9-1b91-4990-b573-645b117a4878","Type":"ContainerStarted","Data":"19e831663ac3c50ca1bc131e1d668040a1b015c05fef9ad35984b9adaf9daf59"} Sep 07 17:09:30.805822 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:30.805758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:30.806017 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:30.805945 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 17:09:30.806017 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:30.805958 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:30.806126 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:30.806019 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:38.805996636 +0000 UTC m=+86.878897951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : secret "insights-runtime-extractor-tls" not found Sep 07 17:09:30.806126 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:30.806108 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 17:09:30.806223 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:30.806144 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:38.806131885 +0000 UTC m=+86.879033199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : configmap "kube-rbac-proxy" not found Sep 07 17:09:30.905800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:30.905769 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:30.963835 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:30.963801 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfwh8" Sep 07 17:09:31.005493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.002125 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-9r6dk" podStartSLOduration=1.914911967 podStartE2EDuration="4.002098455s" totalImagesPullingTime="2.087186488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:30.760026583 +0000 UTC m=+78.832927916" watchObservedRunningTime="2026-09-07 17:09:31.002098455 +0000 UTC m=+79.074999779" Sep 07 17:09:31.094265 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.094229 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:31.201519 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.201476 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:31.268683 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.268585 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:31.352219 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.352176 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:31.428553 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.428507 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:31.529367 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.529259 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:31.628960 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:31.628925 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:32.416432 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:32.416399 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:32.807659 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:32.807624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:33.003863 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:33.003833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:33.401324 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:33.401286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:34.200933 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:34.200898 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:34.410628 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:34.410489 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:34.609972 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:34.609934 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:34.798542 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:34.798508 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:34.999747 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:34.999699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:35.057110 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:35.057083 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:35.613451 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:35.613416 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:35.702835 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:35.702801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:36.244847 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.244812 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:36.423984 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.423919 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:36.535344 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.535227 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:36.636104 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.636068 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:36.648569 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.648540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:36.648698 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.648585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:36.650905 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.650875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d2c8366-312c-4f0d-929b-ad4245f7c236-metrics-tls\") pod \"dns-default-2tgv4\" (UID: \"2d2c8366-312c-4f0d-929b-ad4245f7c236\") " pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:36.651014 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.650941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"image-registry-6cfdfbdc99-nltls\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:36.739278 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.739234 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:36.841139 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.841057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:36.900331 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.900307 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cxf5h\"" Sep 07 17:09:36.907875 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.907852 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:36.910162 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.909831 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq2rt\"" Sep 07 17:09:36.918548 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:36.918528 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:37.187820 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.187740 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfdfbdc99-nltls"] Sep 07 17:09:37.190473 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:37.190444 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5945742_663c_47cb_a119_ef0423cd37f0.slice/crio-7b6ef6d0a0da2a212f704a3275b748acd403a5d1e9080cbad49dc4e196a12fc3 WatchSource:0}: Error finding container 7b6ef6d0a0da2a212f704a3275b748acd403a5d1e9080cbad49dc4e196a12fc3: Status 404 returned error can't find the container with id 7b6ef6d0a0da2a212f704a3275b748acd403a5d1e9080cbad49dc4e196a12fc3 Sep 07 17:09:37.208098 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.208076 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2tgv4"] Sep 07 17:09:37.215681 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:37.215661 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d2c8366_312c_4f0d_929b_ad4245f7c236.slice/crio-600c249b4455a6a42ca16f0659dcd9735e5d6dd7f939a21589f61020fc6c80ab WatchSource:0}: Error finding container 600c249b4455a6a42ca16f0659dcd9735e5d6dd7f939a21589f61020fc6c80ab: Status 404 returned error can't find the container with id 600c249b4455a6a42ca16f0659dcd9735e5d6dd7f939a21589f61020fc6c80ab Sep 07 17:09:37.386514 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.386481 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:37.766869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.766835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tgv4" event={"ID":"2d2c8366-312c-4f0d-929b-ad4245f7c236","Type":"ContainerStarted","Data":"600c249b4455a6a42ca16f0659dcd9735e5d6dd7f939a21589f61020fc6c80ab"} Sep 07 17:09:37.768579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.768551 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" event={"ID":"f5945742-663c-47cb-a119-ef0423cd37f0","Type":"ContainerStarted","Data":"f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e"} Sep 07 17:09:37.768709 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.768594 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" event={"ID":"f5945742-663c-47cb-a119-ef0423cd37f0","Type":"ContainerStarted","Data":"7b6ef6d0a0da2a212f704a3275b748acd403a5d1e9080cbad49dc4e196a12fc3"} Sep 07 17:09:37.768709 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.768695 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:37.793200 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:37.793133 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" podStartSLOduration=65.79311931 podStartE2EDuration="1m5.79311931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:37.79287289 +0000 UTC m=+85.865774223" watchObservedRunningTime="2026-09-07 17:09:37.79311931 +0000 UTC m=+85.866020642" Sep 07 17:09:38.014028 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.013986 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:38.163261 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.163172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:38.165503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.165476 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 17:09:38.176106 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.176083 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d40dd6a8-670a-447b-9d18-7ec8645d13ef-original-pull-secret\") pod \"global-pull-secret-syncer-nmmvg\" (UID: \"d40dd6a8-670a-447b-9d18-7ec8645d13ef\") " pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:38.214539 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.214480 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:38.247163 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.247128 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nmmvg" Sep 07 17:09:38.264351 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.264318 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:38.267041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.267019 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 17:09:38.277120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.277096 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d590b05d-a3ad-434d-bc55-eb1f23c0e214-metrics-certs\") pod \"network-metrics-daemon-cx8hf\" (UID: \"d590b05d-a3ad-434d-bc55-eb1f23c0e214\") " pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:38.365276 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.365244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:38.367710 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.367672 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 17:09:38.378477 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.378445 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 17:09:38.390221 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.390193 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbq8l\" (UniqueName: \"kubernetes.io/projected/4cc3bdcb-b505-48e8-9794-63d6117a1163-kube-api-access-tbq8l\") pod \"network-check-target-vmndt\" (UID: \"4cc3bdcb-b505-48e8-9794-63d6117a1163\") " pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:38.412988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.412947 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:38.536291 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.536261 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mwmqq\"" Sep 07 17:09:38.543126 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.543098 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xmgvk\"" Sep 07 17:09:38.544087 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.544064 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:38.552054 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.552033 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cx8hf" Sep 07 17:09:38.612929 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.612894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:38.809391 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.809367 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:38.869455 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.869427 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:38.869611 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.869521 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:38.869611 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:38.869545 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 17:09:38.869611 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:09:38.869616 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls podName:d45929c7-ef4e-4f12-b694-52cb7b8eb013 nodeName:}" failed. No retries permitted until 2026-09-07 17:09:54.869595591 +0000 UTC m=+102.942496915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hk99l" (UID: "d45929c7-ef4e-4f12-b694-52cb7b8eb013") : secret "insights-runtime-extractor-tls" not found Sep 07 17:09:38.870160 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.870132 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d45929c7-ef4e-4f12-b694-52cb7b8eb013-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:38.936599 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.936559 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nmmvg"] Sep 07 17:09:38.939919 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:38.939892 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd40dd6a8_670a_447b_9d18_7ec8645d13ef.slice/crio-ea59ce2bb6a12cc9c7daa06d858b32737c10f2f0ac94453bc09577b1c630fa30 WatchSource:0}: Error finding container ea59ce2bb6a12cc9c7daa06d858b32737c10f2f0ac94453bc09577b1c630fa30: Status 404 returned error can't find the container with id ea59ce2bb6a12cc9c7daa06d858b32737c10f2f0ac94453bc09577b1c630fa30 Sep 07 17:09:38.970971 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.970939 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vmndt"] Sep 07 17:09:38.972191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:38.972169 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cx8hf"] Sep 07 17:09:38.973908 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:38.973886 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc3bdcb_b505_48e8_9794_63d6117a1163.slice/crio-c04bda0e82ddc927ecc365b512ada75301650a93ebe28e5017c740957af53592 WatchSource:0}: Error finding container c04bda0e82ddc927ecc365b512ada75301650a93ebe28e5017c740957af53592: Status 404 returned error can't find the container with id c04bda0e82ddc927ecc365b512ada75301650a93ebe28e5017c740957af53592 Sep 07 17:09:38.974855 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:38.974834 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd590b05d_a3ad_434d_bc55_eb1f23c0e214.slice/crio-8dd37f2ac2e1eb664d66e91a90cc392409d768828cef749c2152131b8728b291 WatchSource:0}: Error finding container 8dd37f2ac2e1eb664d66e91a90cc392409d768828cef749c2152131b8728b291: Status 404 returned error can't find the container with id 8dd37f2ac2e1eb664d66e91a90cc392409d768828cef749c2152131b8728b291 Sep 07 17:09:39.003423 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.003396 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:39.317349 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.317308 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:39.422461 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.422428 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:39.613480 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.613394 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:39.777839 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.777803 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tgv4" event={"ID":"2d2c8366-312c-4f0d-929b-ad4245f7c236","Type":"ContainerStarted","Data":"87871b868633c851c9f8a543e03696cce147203b254d1593e0a9ad9a9383853f"} Sep 07 17:09:39.778005 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.777869 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tgv4" event={"ID":"2d2c8366-312c-4f0d-929b-ad4245f7c236","Type":"ContainerStarted","Data":"c5c51a53f5db3c5e52ae8019e127724527047c9683e3b74619ca983cdca1e9a8"} Sep 07 17:09:39.778163 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.778149 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:39.780570 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.780541 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vmndt" event={"ID":"4cc3bdcb-b505-48e8-9794-63d6117a1163","Type":"ContainerStarted","Data":"c04bda0e82ddc927ecc365b512ada75301650a93ebe28e5017c740957af53592"} Sep 07 17:09:39.783250 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.783226 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cx8hf" event={"ID":"d590b05d-a3ad-434d-bc55-eb1f23c0e214","Type":"ContainerStarted","Data":"8dd37f2ac2e1eb664d66e91a90cc392409d768828cef749c2152131b8728b291"} Sep 07 17:09:39.784931 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.784910 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nmmvg" event={"ID":"d40dd6a8-670a-447b-9d18-7ec8645d13ef","Type":"ContainerStarted","Data":"ea59ce2bb6a12cc9c7daa06d858b32737c10f2f0ac94453bc09577b1c630fa30"} Sep 07 17:09:39.819057 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.819004 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2tgv4" podStartSLOduration=34.363035643 podStartE2EDuration="35.818985798s" totalImagesPullingTime="1.455950155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:39.818905095 +0000 UTC m=+87.891806429" watchObservedRunningTime="2026-09-07 17:09:39.818985798 +0000 UTC m=+87.891887133" Sep 07 17:09:39.819611 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.819588 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:39.897325 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:39.897220 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:40.189642 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.189533 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:40.254610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.254561 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:40.412329 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.412290 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:40.604938 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.604869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:40.699781 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.699692 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:40.793058 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.792791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cx8hf" event={"ID":"d590b05d-a3ad-434d-bc55-eb1f23c0e214","Type":"ContainerStarted","Data":"0823d50de4ab946b005f0d8f4bca235a8dffddf6ea3352f7ca3f1aeb87ceec81"} Sep 07 17:09:40.793058 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.792858 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cx8hf" event={"ID":"d590b05d-a3ad-434d-bc55-eb1f23c0e214","Type":"ContainerStarted","Data":"13ff22cc8dd36590f84bc8d89e7f55438c28f0cd5359451f863a37f910cae3f2"} Sep 07 17:09:40.813681 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.813428 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cx8hf" podStartSLOduration=67.387750914 podStartE2EDuration="1m8.813408499s" totalImagesPullingTime="1.425657585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:40.812048211 +0000 UTC m=+88.884949545" watchObservedRunningTime="2026-09-07 17:09:40.813408499 +0000 UTC m=+88.886309833" Sep 07 17:09:40.815093 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:40.815039 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:41.005286 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:41.005252 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:41.072225 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:41.072143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:41.205431 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:41.205397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:41.307301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:41.307266 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:41.617137 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:41.617090 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:42.005548 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:42.005510 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:42.233695 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:42.233655 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:42.335145 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:42.335073 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:42.434699 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:42.434660 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:43.806439 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:43.806343 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nmmvg" event={"ID":"d40dd6a8-670a-447b-9d18-7ec8645d13ef","Type":"ContainerStarted","Data":"5bb3fff67e6d42f173adc7730abf43d2d3a7def9776477ecf91aea5233118ea3"} Sep 07 17:09:43.807806 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:43.807781 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vmndt" event={"ID":"4cc3bdcb-b505-48e8-9794-63d6117a1163","Type":"ContainerStarted","Data":"c7b47bd870109b3f612fbf9364f082b0b915bcee021ea7344fb923ca29978f71"} Sep 07 17:09:43.807914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:43.807849 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:09:43.821232 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:43.821189 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nmmvg" podStartSLOduration=67.230589597 podStartE2EDuration="1m11.821174327s" totalImagesPullingTime="4.59058473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:43.820781676 +0000 UTC m=+91.893683009" watchObservedRunningTime="2026-09-07 17:09:43.821174327 +0000 UTC m=+91.894075660" Sep 07 17:09:43.836891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:43.836845 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vmndt" podStartSLOduration=67.285246642 podStartE2EDuration="1m11.836829504s" totalImagesPullingTime="4.551582862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:43.836385489 +0000 UTC m=+91.909286823" watchObservedRunningTime="2026-09-07 17:09:43.836829504 +0000 UTC m=+91.909730868" Sep 07 17:09:44.632707 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:44.632675 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:44.733976 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:44.733945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:49.794572 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:49.794543 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2tgv4" Sep 07 17:09:51.637568 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:51.637537 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:52.729609 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:52.729576 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:52.839834 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:52.839801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:52.929754 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:52.929695 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:53.030391 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:53.030359 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:54.882506 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:54.882474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:54.884751 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:54.884706 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d45929c7-ef4e-4f12-b694-52cb7b8eb013-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hk99l\" (UID: \"d45929c7-ef4e-4f12-b694-52cb7b8eb013\") " pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:55.097299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:55.097267 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hk99l" Sep 07 17:09:55.368418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:55.368385 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hk99l"] Sep 07 17:09:55.372176 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:09:55.372141 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45929c7_ef4e_4f12_b694_52cb7b8eb013.slice/crio-0fc491fd99034f37a7668bedcc0f78dd33fc94185cc2ca7b2b0490913538bc70 WatchSource:0}: Error finding container 0fc491fd99034f37a7668bedcc0f78dd33fc94185cc2ca7b2b0490913538bc70: Status 404 returned error can't find the container with id 0fc491fd99034f37a7668bedcc0f78dd33fc94185cc2ca7b2b0490913538bc70 Sep 07 17:09:55.845998 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:55.845960 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hk99l" event={"ID":"d45929c7-ef4e-4f12-b694-52cb7b8eb013","Type":"ContainerStarted","Data":"2cf8c96a8b1367778c61b86b9c7e42f90f21fb1ed86d56fec63f9f1c8053bc04"} Sep 07 17:09:55.846180 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:55.846021 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hk99l" event={"ID":"d45929c7-ef4e-4f12-b694-52cb7b8eb013","Type":"ContainerStarted","Data":"0fc491fd99034f37a7668bedcc0f78dd33fc94185cc2ca7b2b0490913538bc70"} Sep 07 17:09:56.851376 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:56.851344 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hk99l" event={"ID":"d45929c7-ef4e-4f12-b694-52cb7b8eb013","Type":"ContainerStarted","Data":"87b4d81cbd76a600d6042287a2042e37eab9b964b0664fe7088d45a06757c3af"} Sep 07 17:09:56.913178 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:56.913134 2563 patch_prober.go:28] interesting pod/image-registry-6cfdfbdc99-nltls container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 17:09:56.913341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:56.913205 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" podUID="f5945742-663c-47cb-a119-ef0423cd37f0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 17:09:57.856149 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:57.856112 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hk99l" event={"ID":"d45929c7-ef4e-4f12-b694-52cb7b8eb013","Type":"ContainerStarted","Data":"2bbd0f7148a9f6584ff22cab7a1d944a011ca5a673a611066becac5c6790af58"} Sep 07 17:09:57.876238 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:57.876187 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hk99l" podStartSLOduration=34.120585459 podStartE2EDuration="35.876165806s" totalImagesPullingTime="1.755580347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:09:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:09:57.875519411 +0000 UTC m=+105.948420743" watchObservedRunningTime="2026-09-07 17:09:57.876165806 +0000 UTC m=+105.949067139" Sep 07 17:09:58.776302 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:58.776266 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:09:58.940136 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:58.940107 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:09:59.003857 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:09:59.003784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:01.574648 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:01.574598 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:01.634935 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:01.634902 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:02.603645 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:02.603614 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:02.710247 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:02.710217 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:05.514894 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:05.514860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:05.587744 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:05.587709 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:09.707437 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:09.707402 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:09.790422 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:09.790389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:09.892415 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:09.892375 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:09.995330 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:09.995274 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:10.634626 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:10.634587 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:10.740666 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:10.740631 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:10.832554 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:10.832522 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:11.298372 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.298338 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:11.395781 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.395715 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:11.480224 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.480192 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x"] Sep 07 17:10:11.483187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.483158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" Sep 07 17:10:11.483812 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.483776 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:11.485538 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.485521 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 07 17:10:11.485538 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.485532 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-d7ktd\"" Sep 07 17:10:11.492932 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.492910 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x"] Sep 07 17:10:11.563923 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.563821 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:11.594965 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.594928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1a7b104d-1b06-4a8f-b756-24b8f59fa1de-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-bv47x\" (UID: \"1a7b104d-1b06-4a8f-b756-24b8f59fa1de\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" Sep 07 17:10:11.696247 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.696213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1a7b104d-1b06-4a8f-b756-24b8f59fa1de-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-bv47x\" (UID: \"1a7b104d-1b06-4a8f-b756-24b8f59fa1de\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" Sep 07 17:10:11.698566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.698532 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1a7b104d-1b06-4a8f-b756-24b8f59fa1de-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-bv47x\" (UID: \"1a7b104d-1b06-4a8f-b756-24b8f59fa1de\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" Sep 07 17:10:11.792767 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:11.792712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" Sep 07 17:10:12.069165 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.069136 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x"] Sep 07 17:10:12.072654 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:12.072624 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7b104d_1b06_4a8f_b756_24b8f59fa1de.slice/crio-7d3dc6929477e3c2f951fe79d990363da597cf00e838b19608f1c9f10089e622 WatchSource:0}: Error finding container 7d3dc6929477e3c2f951fe79d990363da597cf00e838b19608f1c9f10089e622: Status 404 returned error can't find the container with id 7d3dc6929477e3c2f951fe79d990363da597cf00e838b19608f1c9f10089e622 Sep 07 17:10:12.335512 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.335484 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:12.433608 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.433580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:12.903704 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.903661 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" event={"ID":"1a7b104d-1b06-4a8f-b756-24b8f59fa1de","Type":"ContainerStarted","Data":"7d3dc6929477e3c2f951fe79d990363da597cf00e838b19608f1c9f10089e622"} Sep 07 17:10:12.955390 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.955357 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77f5955fcd-j2ttz"] Sep 07 17:10:12.959072 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.959029 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:12.974231 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.974208 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:12.977191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.977164 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 07 17:10:12.977328 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.977168 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 07 17:10:12.977328 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.977242 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 07 17:10:12.977911 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.977889 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vmldm\"" Sep 07 17:10:12.978308 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.978240 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 07 17:10:12.978308 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.978288 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 07 17:10:12.978475 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.978325 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 07 17:10:12.978607 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.978574 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 07 17:10:12.979430 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.979410 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77f5955fcd-j2ttz"] Sep 07 17:10:12.985347 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:12.985326 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 07 17:10:13.027924 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.027899 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:13.106429 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.106394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-serving-cert\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.106584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.106432 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xlj5\" (UniqueName: \"kubernetes.io/projected/bbb03ed7-076e-43de-b8a6-3e75db8bed15-kube-api-access-4xlj5\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.106584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.106461 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-service-ca\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.106584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.106536 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-trusted-ca-bundle\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.106584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.106563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-config\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.106789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.106592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-oauth-serving-cert\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.106789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.106647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-oauth-config\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207286 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-config\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207286 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207251 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-oauth-serving-cert\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207508 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207375 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-oauth-config\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207508 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-serving-cert\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207508 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xlj5\" (UniqueName: \"kubernetes.io/projected/bbb03ed7-076e-43de-b8a6-3e75db8bed15-kube-api-access-4xlj5\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207508 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207472 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-service-ca\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207784 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207518 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-trusted-ca-bundle\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.207992 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.207972 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-oauth-serving-cert\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.208129 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.208109 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-config\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.208284 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.208261 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-service-ca\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.208389 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.208372 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-trusted-ca-bundle\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.209927 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.209906 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-serving-cert\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.209927 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.209923 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-oauth-config\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.215500 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.215474 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xlj5\" (UniqueName: \"kubernetes.io/projected/bbb03ed7-076e-43de-b8a6-3e75db8bed15-kube-api-access-4xlj5\") pod \"console-77f5955fcd-j2ttz\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.285702 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.285667 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:13.639579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.639547 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77f5955fcd-j2ttz"] Sep 07 17:10:13.642514 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:13.642490 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb03ed7_076e_43de_b8a6_3e75db8bed15.slice/crio-aec1aed5fe32f3d6dbf07d6d360bc0e40e861a030c488ec9bf401c8b064d63e4 WatchSource:0}: Error finding container aec1aed5fe32f3d6dbf07d6d360bc0e40e861a030c488ec9bf401c8b064d63e4: Status 404 returned error can't find the container with id aec1aed5fe32f3d6dbf07d6d360bc0e40e861a030c488ec9bf401c8b064d63e4 Sep 07 17:10:13.908286 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.908205 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f5955fcd-j2ttz" event={"ID":"bbb03ed7-076e-43de-b8a6-3e75db8bed15","Type":"ContainerStarted","Data":"aec1aed5fe32f3d6dbf07d6d360bc0e40e861a030c488ec9bf401c8b064d63e4"} Sep 07 17:10:13.909638 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.909611 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" event={"ID":"1a7b104d-1b06-4a8f-b756-24b8f59fa1de","Type":"ContainerStarted","Data":"5f086fe531fcff1035ccddf8e759ccf6166ba77304866d152ea7c8b9f1b70e84"} Sep 07 17:10:13.909891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.909871 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" Sep 07 17:10:13.914743 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.914715 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" Sep 07 17:10:13.936436 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:13.936388 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-bv47x" podStartSLOduration=1.635506564 podStartE2EDuration="2.936368706s" totalImagesPullingTime="1.300862142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:10:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:13.933485944 +0000 UTC m=+122.006387277" watchObservedRunningTime="2026-09-07 17:10:13.936368706 +0000 UTC m=+122.009270042" Sep 07 17:10:14.532652 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.532620 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:14.672779 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.672741 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gn26v"] Sep 07 17:10:14.683126 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.683092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:14.693266 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.693236 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.704424 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.704038 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gn26v"] Sep 07 17:10:14.704424 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.704159 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 07 17:10:14.704424 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.704244 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 07 17:10:14.704424 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.704378 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 07 17:10:14.704708 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.704493 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mrfz6\"" Sep 07 17:10:14.704708 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.704606 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 07 17:10:14.705002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.704985 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 07 17:10:14.779545 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.779480 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:14.814083 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.814004 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vmndt" Sep 07 17:10:14.819828 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.819795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.819973 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.819865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.820039 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.819975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.820039 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.820009 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmknx\" (UniqueName: \"kubernetes.io/projected/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-kube-api-access-fmknx\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.920538 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.920442 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.920538 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.920483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmknx\" (UniqueName: \"kubernetes.io/projected/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-kube-api-access-fmknx\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.920538 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.920512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.921059 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.920575 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.921059 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:14.920662 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 07 17:10:14.921059 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:14.920748 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-tls podName:603fcde4-00ab-40ee-b6cd-219c4d4b9a68 nodeName:}" failed. No retries permitted until 2026-09-07 17:10:15.420704877 +0000 UTC m=+123.493606196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-gn26v" (UID: "603fcde4-00ab-40ee-b6cd-219c4d4b9a68") : secret "prometheus-operator-tls" not found Sep 07 17:10:14.921522 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.921491 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.923335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.923313 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:14.931829 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:14.931807 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmknx\" (UniqueName: \"kubernetes.io/projected/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-kube-api-access-fmknx\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:15.054034 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:15.053998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:15.262430 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:15.262392 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cfdfbdc99-nltls"] Sep 07 17:10:15.392552 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:15.392491 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:15.425915 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:15.425883 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:15.428517 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:15.428491 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/603fcde4-00ab-40ee-b6cd-219c4d4b9a68-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gn26v\" (UID: \"603fcde4-00ab-40ee-b6cd-219c4d4b9a68\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:15.481007 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:15.480967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:15.605408 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:15.605330 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" Sep 07 17:10:16.656056 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:16.655983 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gn26v"] Sep 07 17:10:16.713942 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:16.713903 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603fcde4_00ab_40ee_b6cd_219c4d4b9a68.slice/crio-f7938b540d221c969c6548ca2ef782449be593ec779c04bd8cd6f4890935bc89 WatchSource:0}: Error finding container f7938b540d221c969c6548ca2ef782449be593ec779c04bd8cd6f4890935bc89: Status 404 returned error can't find the container with id f7938b540d221c969c6548ca2ef782449be593ec779c04bd8cd6f4890935bc89 Sep 07 17:10:16.922079 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:16.921984 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f5955fcd-j2ttz" event={"ID":"bbb03ed7-076e-43de-b8a6-3e75db8bed15","Type":"ContainerStarted","Data":"d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b"} Sep 07 17:10:16.923066 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:16.923041 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" event={"ID":"603fcde4-00ab-40ee-b6cd-219c4d4b9a68","Type":"ContainerStarted","Data":"f7938b540d221c969c6548ca2ef782449be593ec779c04bd8cd6f4890935bc89"} Sep 07 17:10:16.942803 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:16.942706 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77f5955fcd-j2ttz" podStartSLOduration=1.8407339139999999 podStartE2EDuration="4.942690555s" totalImagesPullingTime="3.101956641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:16.942055547 +0000 UTC m=+125.014956884" watchObservedRunningTime="2026-09-07 17:10:16.942690555 +0000 UTC m=+125.015591888" Sep 07 17:10:17.928808 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:17.928751 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" event={"ID":"603fcde4-00ab-40ee-b6cd-219c4d4b9a68","Type":"ContainerStarted","Data":"c85128fb7eb666f31f8f4792743a0b149b0d46aa3854ceab8db47f3dba6312f8"} Sep 07 17:10:17.929208 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:17.928846 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" event={"ID":"603fcde4-00ab-40ee-b6cd-219c4d4b9a68","Type":"ContainerStarted","Data":"0749a5a6cfe6fe73837c5085edf031687d1a510c4c6456334ab17b4f55f91d49"} Sep 07 17:10:17.958908 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:17.958845 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-gn26v" podStartSLOduration=2.889313677 podStartE2EDuration="3.958825316s" totalImagesPullingTime="1.069511639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:17.957598316 +0000 UTC m=+126.030499650" watchObservedRunningTime="2026-09-07 17:10:17.958825316 +0000 UTC m=+126.031726651" Sep 07 17:10:18.187746 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:18.187647 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:19.772809 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:19.772775 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:20.043617 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.043525 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:20.169801 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.169767 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk"] Sep 07 17:10:20.171884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.171869 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.175040 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.175015 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 07 17:10:20.175150 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.175081 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 07 17:10:20.175406 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.175387 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 07 17:10:20.175460 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.175387 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lccqf\"" Sep 07 17:10:20.180199 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.180176 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:20.200395 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.200376 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk"] Sep 07 17:10:20.209403 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.209385 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ndnfg"] Sep 07 17:10:20.211352 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.211337 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.213325 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.213307 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 07 17:10:20.213412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.213347 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 07 17:10:20.213487 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.213421 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9xkvj\"" Sep 07 17:10:20.213616 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.213601 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 07 17:10:20.266262 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.266236 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.266414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.266265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.266414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.266345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.266414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.266372 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.266414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.266401 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.266540 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.266420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pnhp\" (UniqueName: \"kubernetes.io/projected/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-api-access-9pnhp\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.279524 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.279499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:20.367377 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367304 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-root\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.367377 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367338 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.367377 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367356 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-accelerators-collector-config\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.367596 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-sys\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.367596 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367420 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.367596 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367469 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2f8587-20ab-4545-b59a-7efe60c32af2-metrics-client-ca\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.367768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psh4p\" (UniqueName: \"kubernetes.io/projected/2e2f8587-20ab-4545-b59a-7efe60c32af2-kube-api-access-psh4p\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.367768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367647 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.367768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367676 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.367768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367711 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pnhp\" (UniqueName: \"kubernetes.io/projected/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-api-access-9pnhp\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.367768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367766 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-wtmp\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.368010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.368010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.368010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-tls\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.368010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.367876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-textfile\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.368198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.368044 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.368252 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.368196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.368308 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.368281 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.370299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.370277 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.370378 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.370310 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.379560 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.379542 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pnhp\" (UniqueName: \"kubernetes.io/projected/55c4ac9c-51d7-4cb9-aa23-6f9c45c83748-kube-api-access-9pnhp\") pod \"kube-state-metrics-74cb57df7-hf5lk\" (UID: \"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.468618 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468587 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-tls\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468618 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468622 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-textfile\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468656 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-root\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468690 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-accelerators-collector-config\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468754 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-root\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468775 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-sys\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:20.468784 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468801 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2f8587-20ab-4545-b59a-7efe60c32af2-metrics-client-ca\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.468869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468838 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psh4p\" (UniqueName: \"kubernetes.io/projected/2e2f8587-20ab-4545-b59a-7efe60c32af2-kube-api-access-psh4p\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.469279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468880 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-wtmp\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.469279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.468993 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-textfile\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.469279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.469027 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-wtmp\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.469279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.469136 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e2f8587-20ab-4545-b59a-7efe60c32af2-sys\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.469279 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:20.469189 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-tls podName:2e2f8587-20ab-4545-b59a-7efe60c32af2 nodeName:}" failed. No retries permitted until 2026-09-07 17:10:20.969165328 +0000 UTC m=+129.042066654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-tls") pod "node-exporter-ndnfg" (UID: "2e2f8587-20ab-4545-b59a-7efe60c32af2") : secret "node-exporter-tls" not found Sep 07 17:10:20.469562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.469497 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2f8587-20ab-4545-b59a-7efe60c32af2-metrics-client-ca\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.469562 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.469530 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-accelerators-collector-config\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.471222 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.471198 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.481661 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.481641 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psh4p\" (UniqueName: \"kubernetes.io/projected/2e2f8587-20ab-4545-b59a-7efe60c32af2-kube-api-access-psh4p\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.481778 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.481747 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" Sep 07 17:10:20.498926 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.498904 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:20.738858 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.738812 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:20.774363 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.772169 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk"] Sep 07 17:10:20.779058 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:20.779028 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c4ac9c_51d7_4cb9_aa23_6f9c45c83748.slice/crio-47abfbb475b0b9621e2d115d24d8d0e653a9685166882100ec7362b1aaed95ee WatchSource:0}: Error finding container 47abfbb475b0b9621e2d115d24d8d0e653a9685166882100ec7362b1aaed95ee: Status 404 returned error can't find the container with id 47abfbb475b0b9621e2d115d24d8d0e653a9685166882100ec7362b1aaed95ee Sep 07 17:10:20.841425 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.841389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:20.940528 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.940496 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" event={"ID":"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748","Type":"ContainerStarted","Data":"47abfbb475b0b9621e2d115d24d8d0e653a9685166882100ec7362b1aaed95ee"} Sep 07 17:10:20.973615 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.973583 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-tls\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.976017 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.975991 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2f8587-20ab-4545-b59a-7efe60c32af2-node-exporter-tls\") pod \"node-exporter-ndnfg\" (UID: \"2e2f8587-20ab-4545-b59a-7efe60c32af2\") " pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:20.997342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:20.997261 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:21.121272 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.121236 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ndnfg" Sep 07 17:10:21.133206 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:21.133100 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2f8587_20ab_4545_b59a_7efe60c32af2.slice/crio-e7775f8a6400013b9479e6d34c4f27045a573074714f163511cf3d87507c6fbf WatchSource:0}: Error finding container e7775f8a6400013b9479e6d34c4f27045a573074714f163511cf3d87507c6fbf: Status 404 returned error can't find the container with id e7775f8a6400013b9479e6d34c4f27045a573074714f163511cf3d87507c6fbf Sep 07 17:10:21.157883 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.157846 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:10:21.161455 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.161412 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.161573 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.161523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:21.166659 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.166628 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 07 17:10:21.166784 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.166697 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 07 17:10:21.166890 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.166872 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 07 17:10:21.166967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.166944 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 07 17:10:21.167087 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.167070 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 07 17:10:21.167200 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.167176 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-m5kbw\"" Sep 07 17:10:21.167322 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.167286 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 07 17:10:21.167486 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.167473 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 07 17:10:21.167670 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.167652 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 07 17:10:21.168204 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.168186 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 07 17:10:21.196770 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.196230 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:10:21.275749 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275662 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275749 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275712 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275762 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275792 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275824 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275856 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-out\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275929 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-web-config\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.275981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.275956 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.276335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.276017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.276335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.276068 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdhlh\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-kube-api-access-bdhlh\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.376810 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.376773 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-out\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.376988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.376832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.376988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.376868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-web-config\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.376988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.376903 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.376988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.376928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.376988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.376952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdhlh\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-kube-api-access-bdhlh\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.376988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.376986 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.377301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.377025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.377301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.377049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.377301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.377075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.377301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.377097 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.377301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.377111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.377301 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.377127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.378153 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.377841 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.378264 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.378238 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.379770 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.379743 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.379874 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:21.379859 2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 07 17:10:21.379933 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:21.379916 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls podName:aa5e7d66-6765-4f04-954f-c7cddbead03e nodeName:}" failed. No retries permitted until 2026-09-07 17:10:21.879898077 +0000 UTC m=+129.952799390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e") : secret "alertmanager-main-tls" not found Sep 07 17:10:21.380074 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.380049 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-web-config\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.380294 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.380252 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.381184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.381142 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.381773 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.381712 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-out\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.382908 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.382864 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.383004 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.382962 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.383073 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.383016 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.383434 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.383409 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.397156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.397113 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdhlh\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-kube-api-access-bdhlh\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.566953 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.566872 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:21.880909 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.880858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.883026 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.882998 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:21.921313 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.921212 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:21.949988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.949866 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" event={"ID":"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748","Type":"ContainerStarted","Data":"34d6d9818d1c9875c03752bf6ab3ccc62f1c26afb825d7c1eb09bf3381e77136"} Sep 07 17:10:21.949988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.949934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" event={"ID":"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748","Type":"ContainerStarted","Data":"909521b0d6a66c65791891453f66a62fd7f4f1d053e7d358131e35ad19cc80da"} Sep 07 17:10:21.953487 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:21.953399 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ndnfg" event={"ID":"2e2f8587-20ab-4545-b59a-7efe60c32af2","Type":"ContainerStarted","Data":"e7775f8a6400013b9479e6d34c4f27045a573074714f163511cf3d87507c6fbf"} Sep 07 17:10:22.075001 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.074866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:10:22.382989 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.382820 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:10:22.396705 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:22.396675 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5e7d66_6765_4f04_954f_c7cddbead03e.slice/crio-490a2e72ae77b2f0ad476b39d32c4a223abf97d0bbb32dfb728b76b68c0fc2cd WatchSource:0}: Error finding container 490a2e72ae77b2f0ad476b39d32c4a223abf97d0bbb32dfb728b76b68c0fc2cd: Status 404 returned error can't find the container with id 490a2e72ae77b2f0ad476b39d32c4a223abf97d0bbb32dfb728b76b68c0fc2cd Sep 07 17:10:22.527719 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.527685 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:22.628844 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.628758 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:22.960304 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.960216 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" event={"ID":"55c4ac9c-51d7-4cb9-aa23-6f9c45c83748","Type":"ContainerStarted","Data":"347a27eec255c4b308e1ba53449c4e50cee42b58a70c588895bfa2885e7d1a8e"} Sep 07 17:10:22.962035 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.962010 2563 generic.go:356] "Generic (PLEG): container finished" podID="2e2f8587-20ab-4545-b59a-7efe60c32af2" containerID="fa42b8abdfabf05f2387826dd5de2bc87b26d9cd94542e52eac7d8faf7b42302" exitCode=0 Sep 07 17:10:22.962156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.962100 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ndnfg" event={"ID":"2e2f8587-20ab-4545-b59a-7efe60c32af2","Type":"ContainerDied","Data":"fa42b8abdfabf05f2387826dd5de2bc87b26d9cd94542e52eac7d8faf7b42302"} Sep 07 17:10:22.963580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:22.963555 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerStarted","Data":"490a2e72ae77b2f0ad476b39d32c4a223abf97d0bbb32dfb728b76b68c0fc2cd"} Sep 07 17:10:23.041153 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.041103 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hf5lk" podStartSLOduration=2.041693957 podStartE2EDuration="3.041086181s" totalImagesPullingTime="999.392224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:23.005563498 +0000 UTC m=+131.078464831" watchObservedRunningTime="2026-09-07 17:10:23.041086181 +0000 UTC m=+131.113987514" Sep 07 17:10:23.189980 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.189943 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:23.265881 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.265843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:23.286786 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.286761 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:23.286922 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.286824 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:23.292172 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.292151 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:23.366941 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.366909 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:23.561479 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.561393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:23.969033 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.968947 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerID="a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db" exitCode=0 Sep 07 17:10:23.969425 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.969033 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db"} Sep 07 17:10:23.971385 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.971363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ndnfg" event={"ID":"2e2f8587-20ab-4545-b59a-7efe60c32af2","Type":"ContainerStarted","Data":"810db5c5265fc9e928ac4693609e4f63d103801578942e7d44a02c63f578568c"} Sep 07 17:10:23.971467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.971410 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ndnfg" event={"ID":"2e2f8587-20ab-4545-b59a-7efe60c32af2","Type":"ContainerStarted","Data":"12e804d98e31494248e81bd5d7569e6df72e34ca061bd7070977a59047ad4709"} Sep 07 17:10:23.975393 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:23.975374 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:10:24.035641 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.035520 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ndnfg" podStartSLOduration=2.7576006189999998 podStartE2EDuration="4.035500537s" totalImagesPullingTime="1.214426999s" totalInitContainerRuntime="63.472919ms" isStatefulPod=true podCreationTimestamp="2026-09-07 17:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:24.034494099 +0000 UTC m=+132.107395432" watchObservedRunningTime="2026-09-07 17:10:24.035500537 +0000 UTC m=+132.108401871" Sep 07 17:10:24.257119 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.257092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:24.336451 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.336415 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:24.458982 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.458947 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:24.562549 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.562465 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:24.581579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.581550 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-64868b84df-pqxn4"] Sep 07 17:10:24.584807 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.584788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.587618 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.587581 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 07 17:10:24.588068 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.588048 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6nw58\"" Sep 07 17:10:24.588184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.588070 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 07 17:10:24.588255 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.588191 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9el7udmvio6c4\"" Sep 07 17:10:24.588471 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.588451 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 07 17:10:24.588766 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.588745 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 07 17:10:24.597427 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.597406 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64868b84df-pqxn4"] Sep 07 17:10:24.710917 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.710883 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8313e248-1c50-459e-b0fb-c7743aa20db0-audit-log\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.711094 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.710970 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8313e248-1c50-459e-b0fb-c7743aa20db0-metrics-server-audit-profiles\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.711094 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.711027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-secret-metrics-server-tls\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.711094 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.711065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-secret-metrics-server-client-certs\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.711277 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.711129 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-client-ca-bundle\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.711277 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.711204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8313e248-1c50-459e-b0fb-c7743aa20db0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.711277 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.711265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9455g\" (UniqueName: \"kubernetes.io/projected/8313e248-1c50-459e-b0fb-c7743aa20db0-kube-api-access-9455g\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.798860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.798823 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:24.811874 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.811848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9455g\" (UniqueName: \"kubernetes.io/projected/8313e248-1c50-459e-b0fb-c7743aa20db0-kube-api-access-9455g\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812035 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.811902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8313e248-1c50-459e-b0fb-c7743aa20db0-audit-log\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812035 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.811957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8313e248-1c50-459e-b0fb-c7743aa20db0-metrics-server-audit-profiles\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812148 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.812033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-secret-metrics-server-tls\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812148 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.812070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-secret-metrics-server-client-certs\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812148 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.812096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-client-ca-bundle\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812148 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.812125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8313e248-1c50-459e-b0fb-c7743aa20db0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.812339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8313e248-1c50-459e-b0fb-c7743aa20db0-audit-log\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.812918 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.812859 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8313e248-1c50-459e-b0fb-c7743aa20db0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.813313 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.813290 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8313e248-1c50-459e-b0fb-c7743aa20db0-metrics-server-audit-profiles\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.815011 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.814990 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-client-ca-bundle\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.815513 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.815491 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-secret-metrics-server-tls\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.815786 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.815766 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8313e248-1c50-459e-b0fb-c7743aa20db0-secret-metrics-server-client-certs\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.831898 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.831870 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9455g\" (UniqueName: \"kubernetes.io/projected/8313e248-1c50-459e-b0fb-c7743aa20db0-kube-api-access-9455g\") pod \"metrics-server-64868b84df-pqxn4\" (UID: \"8313e248-1c50-459e-b0fb-c7743aa20db0\") " pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.878685 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.878642 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:24.897493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.897460 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:24.952608 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:24.951325 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:25.070214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.070122 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:25.189412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.189377 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64868b84df-pqxn4"] Sep 07 17:10:25.285625 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:25.285592 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8313e248_1c50_459e_b0fb_c7743aa20db0.slice/crio-4eed6f1e62c13e5120118f77a10895ff2f7dab0b478f88060ad798b9e197d495 WatchSource:0}: Error finding container 4eed6f1e62c13e5120118f77a10895ff2f7dab0b478f88060ad798b9e197d495: Status 404 returned error can't find the container with id 4eed6f1e62c13e5120118f77a10895ff2f7dab0b478f88060ad798b9e197d495 Sep 07 17:10:25.986981 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.986900 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerStarted","Data":"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df"} Sep 07 17:10:25.987137 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.986982 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerStarted","Data":"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0"} Sep 07 17:10:25.987137 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.987022 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerStarted","Data":"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db"} Sep 07 17:10:25.987137 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.987060 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerStarted","Data":"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e"} Sep 07 17:10:25.987137 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.987095 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerStarted","Data":"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05"} Sep 07 17:10:25.988366 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:25.988337 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" event={"ID":"8313e248-1c50-459e-b0fb-c7743aa20db0","Type":"ContainerStarted","Data":"4eed6f1e62c13e5120118f77a10895ff2f7dab0b478f88060ad798b9e197d495"} Sep 07 17:10:26.479922 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.479883 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d6886fd7f-dp44v"] Sep 07 17:10:26.483555 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.483529 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.499837 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.499804 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6886fd7f-dp44v"] Sep 07 17:10:26.533074 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.533044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:26.629553 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.629513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-oauth-serving-cert\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.629553 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.629555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9wr\" (UniqueName: \"kubernetes.io/projected/34314299-5ca7-46ec-9159-8568af291e23-kube-api-access-nw9wr\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.629792 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.629602 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-serving-cert\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.629792 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.629670 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-oauth-config\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.629792 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.629717 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-trusted-ca-bundle\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.629913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.629795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-console-config\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.629913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.629829 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-service-ca\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.730694 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.730608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-oauth-config\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.730694 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.730664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-trusted-ca-bundle\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.730936 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.730755 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-console-config\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.730936 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.730802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-service-ca\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.730936 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.730840 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-oauth-serving-cert\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.730936 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.730871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9wr\" (UniqueName: \"kubernetes.io/projected/34314299-5ca7-46ec-9159-8568af291e23-kube-api-access-nw9wr\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.730936 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.730918 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-serving-cert\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.731508 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.731470 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-console-config\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.731754 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.731715 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-trusted-ca-bundle\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.731853 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.731766 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-service-ca\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.731853 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.731766 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-oauth-serving-cert\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.733607 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.733585 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-oauth-config\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.733789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.733768 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-serving-cert\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.742341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.742325 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9wr\" (UniqueName: \"kubernetes.io/projected/34314299-5ca7-46ec-9159-8568af291e23-kube-api-access-nw9wr\") pod \"console-7d6886fd7f-dp44v\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.795184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.794936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:26.911894 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.911862 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:26.983159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.983050 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:26.996154 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:26.996117 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerStarted","Data":"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e"} Sep 07 17:10:27.000045 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:27.000015 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" event={"ID":"8313e248-1c50-459e-b0fb-c7743aa20db0","Type":"ContainerStarted","Data":"58b1f1c078c312bb2ded1679b05734cd10cf0707a75070f495965ec731a64483"} Sep 07 17:10:27.031497 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:27.031449 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.620603568 podStartE2EDuration="6.031433876s" totalImagesPullingTime="3.391709501s" totalInitContainerRuntime="19.120807ms" isStatefulPod=false podCreationTimestamp="2026-09-07 17:10:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:27.030277547 +0000 UTC m=+135.103178881" watchObservedRunningTime="2026-09-07 17:10:27.031433876 +0000 UTC m=+135.104335209" Sep 07 17:10:27.065083 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:27.065043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:27.090080 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:27.089997 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" podStartSLOduration=1.6396182590000001 podStartE2EDuration="3.089972276s" totalImagesPullingTime="1.450354017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:27.055503892 +0000 UTC m=+135.128405226" watchObservedRunningTime="2026-09-07 17:10:27.089972276 +0000 UTC m=+135.162873611" Sep 07 17:10:27.092484 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:27.092441 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6886fd7f-dp44v"] Sep 07 17:10:27.095173 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:10:27.095149 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34314299_5ca7_46ec_9159_8568af291e23.slice/crio-e26ba889f43e15e1f7e0828239c3cd1e321489524befc0e75c81322b7df11011 WatchSource:0}: Error finding container e26ba889f43e15e1f7e0828239c3cd1e321489524befc0e75c81322b7df11011: Status 404 returned error can't find the container with id e26ba889f43e15e1f7e0828239c3cd1e321489524befc0e75c81322b7df11011 Sep 07 17:10:27.323670 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:27.323634 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:27.393652 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:27.393618 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:28.005307 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:28.005272 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6886fd7f-dp44v" event={"ID":"34314299-5ca7-46ec-9159-8568af291e23","Type":"ContainerStarted","Data":"1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee"} Sep 07 17:10:28.005683 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:28.005329 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6886fd7f-dp44v" event={"ID":"34314299-5ca7-46ec-9159-8568af291e23","Type":"ContainerStarted","Data":"e26ba889f43e15e1f7e0828239c3cd1e321489524befc0e75c81322b7df11011"} Sep 07 17:10:28.030993 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:28.030942 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d6886fd7f-dp44v" podStartSLOduration=2.030927114 podStartE2EDuration="2.030927114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:10:28.029642502 +0000 UTC m=+136.102543836" watchObservedRunningTime="2026-09-07 17:10:28.030927114 +0000 UTC m=+136.103828447" Sep 07 17:10:30.307635 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:30.307600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:30.396341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:30.396313 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:30.497567 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:30.497534 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:33.474683 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:33.474655 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2l6hx_d3b8ac2d-79b3-4163-898f-aa94677f513b/serve-healthcheck-canary/0.log" Sep 07 17:10:36.795343 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:36.795291 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:36.795763 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:36.795374 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:36.800080 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:36.800057 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:37.041516 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:37.041487 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:39.918595 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:39.918542 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34314299-5ca7-46ec-9159-8568af291e23\\\"},\\\"status\\\":{\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"},\\\"containerID\\\":\\\"cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"console\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T17:10:27Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000210000],\\\"uid\\\":1000210000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/serving-cert\\\",\\\"name\\\":\\\"console-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-config\\\",\\\"name\\\":\\\"console-oauth-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/console-config\\\",\\\"name\\\":\\\"console-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/service-ca\\\",\\\"name\\\":\\\"service-ca\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted/pem\\\",\\\"name\\\":\\\"trusted-ca-bundle\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-serving-cert\\\",\\\"name\\\":\\\"oauth-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-nw9wr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-console\"/\"console-7d6886fd7f-dp44v\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:39.940987 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:39.940926 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34314299-5ca7-46ec-9159-8568af291e23\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"},\\\"containerID\\\":\\\"cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"console\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T17:10:27Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000210000],\\\"uid\\\":1000210000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/serving-cert\\\",\\\"name\\\":\\\"console-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-config\\\",\\\"name\\\":\\\"console-oauth-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/console-config\\\",\\\"name\\\":\\\"console-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/service-ca\\\",\\\"name\\\":\\\"service-ca\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted/pem\\\",\\\"name\\\":\\\"trusted-ca-bundle\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-serving-cert\\\",\\\"name\\\":\\\"oauth-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-nw9wr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-console\"/\"console-7d6886fd7f-dp44v\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:40.224776 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.224692 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:40.281356 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.281324 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" podUID="f5945742-663c-47cb-a119-ef0423cd37f0" containerName="registry" containerID="cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e" gracePeriod=30 Sep 07 17:10:40.315503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.315476 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:40.439074 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.439019 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:40.525815 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.525790 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:10:40.545739 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.545704 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-trusted-ca\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.545837 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.545744 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-bound-sa-token\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.545837 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.545769 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-registry-certificates\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.545925 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.545880 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-image-registry-private-configuration\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.545925 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.545918 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.546193 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.546166 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:10:40.546279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.546174 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:10:40.548162 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.548137 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:10:40.548256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.548182 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:10:40.548256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.548231 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:10:40.553555 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:40.553512 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34314299-5ca7-46ec-9159-8568af291e23\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"},\\\"containerID\\\":\\\"cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"console\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T17:10:27Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000210000],\\\"uid\\\":1000210000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/serving-cert\\\",\\\"name\\\":\\\"console-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-config\\\",\\\"name\\\":\\\"console-oauth-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/console-config\\\",\\\"name\\\":\\\"console-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/service-ca\\\",\\\"name\\\":\\\"service-ca\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted/pem\\\",\\\"name\\\":\\\"trusted-ca-bundle\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-serving-cert\\\",\\\"name\\\":\\\"oauth-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-nw9wr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-console\"/\"console-7d6886fd7f-dp44v\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:40.562067 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.562037 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:40.601757 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:40.601687 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f5945742-663c-47cb-a119-ef0423cd37f0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"},\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"registry\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-07T17:09:37Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000290000],\\\"uid\\\":1000290000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/cloud\\\",\\\"name\\\":\\\"image-registry-private-configuration\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/secrets\\\",\\\"name\\\":\\\"registry-tls\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted\\\",\\\"name\\\":\\\"ca-trust-extracted\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/source/anchors\\\",\\\"name\\\":\\\"registry-certificates\\\"},{\\\"mountPath\\\":\\\"/usr/share/pki/ca-trust-source\\\",\\\"name\\\":\\\"trusted-ca\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet/\\\",\\\"name\\\":\\\"installation-pull-secrets\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/openshift/serviceaccount\\\",\\\"name\\\":\\\"bound-sa-token\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ntfs8\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-image-registry\"/\"image-registry-6cfdfbdc99-nltls\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:10:40.647225 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647199 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntfs8\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-kube-api-access-ntfs8\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.647306 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647228 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5945742-663c-47cb-a119-ef0423cd37f0-ca-trust-extracted\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.647306 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647256 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-installation-pull-secrets\") pod \"f5945742-663c-47cb-a119-ef0423cd37f0\" (UID: \"f5945742-663c-47cb-a119-ef0423cd37f0\") " Sep 07 17:10:40.647422 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647406 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-image-registry-private-configuration\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:40.647455 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647428 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-registry-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:40.647455 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647441 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-trusted-ca\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:40.647520 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647451 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-bound-sa-token\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:40.647520 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.647466 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5945742-663c-47cb-a119-ef0423cd37f0-registry-certificates\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:40.649282 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.649259 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-kube-api-access-ntfs8" (OuterVolumeSpecName: "kube-api-access-ntfs8") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "kube-api-access-ntfs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:10:40.649344 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.649260 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:10:40.654312 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.654289 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5945742-663c-47cb-a119-ef0423cd37f0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f5945742-663c-47cb-a119-ef0423cd37f0" (UID: "f5945742-663c-47cb-a119-ef0423cd37f0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:10:40.748239 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.748198 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntfs8\" (UniqueName: \"kubernetes.io/projected/f5945742-663c-47cb-a119-ef0423cd37f0-kube-api-access-ntfs8\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:40.748239 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.748232 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5945742-663c-47cb-a119-ef0423cd37f0-ca-trust-extracted\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:40.748239 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:40.748245 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5945742-663c-47cb-a119-ef0423cd37f0-installation-pull-secrets\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:10:41.057214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.057181 2563 generic.go:356] "Generic (PLEG): container finished" podID="f5945742-663c-47cb-a119-ef0423cd37f0" containerID="f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e" exitCode=0 Sep 07 17:10:41.057624 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.057234 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:10:41.057624 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.057260 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" event={"ID":"f5945742-663c-47cb-a119-ef0423cd37f0","Type":"ContainerDied","Data":"f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e"} Sep 07 17:10:41.057624 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.057328 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" event={"ID":"f5945742-663c-47cb-a119-ef0423cd37f0","Type":"ContainerDied","Data":"7b6ef6d0a0da2a212f704a3275b748acd403a5d1e9080cbad49dc4e196a12fc3"} Sep 07 17:10:41.057624 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.057347 2563 scope.go:117] "RemoveContainer" containerID="f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e" Sep 07 17:10:41.065636 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.065617 2563 scope.go:117] "RemoveContainer" containerID="f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e" Sep 07 17:10:41.065926 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:41.065902 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\": container with ID starting with f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e not found: ID does not exist" containerID="f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e" Sep 07 17:10:41.065975 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.065935 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e"} err="failed to get container status \"f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\": rpc error: code = NotFound desc = could not find container \"f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\": container with ID starting with f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e not found: ID does not exist" Sep 07 17:10:41.073838 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:41.073797 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34314299-5ca7-46ec-9159-8568af291e23\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"},\\\"containerID\\\":\\\"cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"console\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T17:10:27Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000210000],\\\"uid\\\":1000210000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/serving-cert\\\",\\\"name\\\":\\\"console-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-config\\\",\\\"name\\\":\\\"console-oauth-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/console-config\\\",\\\"name\\\":\\\"console-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/service-ca\\\",\\\"name\\\":\\\"service-ca\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted/pem\\\",\\\"name\\\":\\\"trusted-ca-bundle\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-serving-cert\\\",\\\"name\\\":\\\"oauth-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-nw9wr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-console\"/\"console-7d6886fd7f-dp44v\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:41.097966 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:41.097917 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f5945742-663c-47cb-a119-ef0423cd37f0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"},\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"registry\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-07T17:09:37Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000290000],\\\"uid\\\":1000290000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/cloud\\\",\\\"name\\\":\\\"image-registry-private-configuration\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/secrets\\\",\\\"name\\\":\\\"registry-tls\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted\\\",\\\"name\\\":\\\"ca-trust-extracted\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/source/anchors\\\",\\\"name\\\":\\\"registry-certificates\\\"},{\\\"mountPath\\\":\\\"/usr/share/pki/ca-trust-source\\\",\\\"name\\\":\\\"trusted-ca\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet/\\\",\\\"name\\\":\\\"installation-pull-secrets\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/openshift/serviceaccount\\\",\\\"name\\\":\\\"bound-sa-token\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ntfs8\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-image-registry\"/\"image-registry-6cfdfbdc99-nltls\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:10:41.120866 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:41.120808 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f5945742-663c-47cb-a119-ef0423cd37f0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"},\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"registry\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-07T17:09:37Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000290000],\\\"uid\\\":1000290000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/cloud\\\",\\\"name\\\":\\\"image-registry-private-configuration\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/secrets\\\",\\\"name\\\":\\\"registry-tls\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted\\\",\\\"name\\\":\\\"ca-trust-extracted\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/source/anchors\\\",\\\"name\\\":\\\"registry-certificates\\\"},{\\\"mountPath\\\":\\\"/usr/share/pki/ca-trust-source\\\",\\\"name\\\":\\\"trusted-ca\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet/\\\",\\\"name\\\":\\\"installation-pull-secrets\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/openshift/serviceaccount\\\",\\\"name\\\":\\\"bound-sa-token\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ntfs8\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-image-registry\"/\"image-registry-6cfdfbdc99-nltls\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:10:41.146288 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:41.146243 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34314299-5ca7-46ec-9159-8568af291e23\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"},\\\"containerID\\\":\\\"cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"console\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T17:10:27Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000210000],\\\"uid\\\":1000210000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/serving-cert\\\",\\\"name\\\":\\\"console-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-config\\\",\\\"name\\\":\\\"console-oauth-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/console-config\\\",\\\"name\\\":\\\"console-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/service-ca\\\",\\\"name\\\":\\\"service-ca\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted/pem\\\",\\\"name\\\":\\\"trusted-ca-bundle\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-serving-cert\\\",\\\"name\\\":\\\"oauth-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-nw9wr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-console\"/\"console-7d6886fd7f-dp44v\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:41.690327 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:41.690299 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:42.349914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:42.349883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:42.445926 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:42.445871 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34314299-5ca7-46ec-9159-8568af291e23\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"},\\\"containerID\\\":\\\"cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"console\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T17:10:27Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000210000],\\\"uid\\\":1000210000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/serving-cert\\\",\\\"name\\\":\\\"console-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-config\\\",\\\"name\\\":\\\"console-oauth-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/console-config\\\",\\\"name\\\":\\\"console-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/service-ca\\\",\\\"name\\\":\\\"service-ca\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted/pem\\\",\\\"name\\\":\\\"trusted-ca-bundle\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-serving-cert\\\",\\\"name\\\":\\\"oauth-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-nw9wr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-console\"/\"console-7d6886fd7f-dp44v\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:42.452759 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:42.452697 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:42.469695 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:42.469646 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f5945742-663c-47cb-a119-ef0423cd37f0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"},\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"registry\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-07T17:09:37Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000290000],\\\"uid\\\":1000290000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/cloud\\\",\\\"name\\\":\\\"image-registry-private-configuration\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/secrets\\\",\\\"name\\\":\\\"registry-tls\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted\\\",\\\"name\\\":\\\"ca-trust-extracted\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/source/anchors\\\",\\\"name\\\":\\\"registry-certificates\\\"},{\\\"mountPath\\\":\\\"/usr/share/pki/ca-trust-source\\\",\\\"name\\\":\\\"trusted-ca\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet/\\\",\\\"name\\\":\\\"installation-pull-secrets\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/openshift/serviceaccount\\\",\\\"name\\\":\\\"bound-sa-token\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ntfs8\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-image-registry\"/\"image-registry-6cfdfbdc99-nltls\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:10:44.897921 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:44.897881 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:44.898416 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:44.897968 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:10:52.445971 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:52.445912 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f5945742-663c-47cb-a119-ef0423cd37f0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"},\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"registry\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"256Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://f4f1d1f5d9d8b163f28d79eeacaf778aa178de0f895ea8ae2486d169e3ae950e\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-07T17:10:40Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-07T17:09:37Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000290000],\\\"uid\\\":1000290000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/cloud\\\",\\\"name\\\":\\\"image-registry-private-configuration\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/secrets\\\",\\\"name\\\":\\\"registry-tls\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted\\\",\\\"name\\\":\\\"ca-trust-extracted\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/source/anchors\\\",\\\"name\\\":\\\"registry-certificates\\\"},{\\\"mountPath\\\":\\\"/usr/share/pki/ca-trust-source\\\",\\\"name\\\":\\\"trusted-ca\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet/\\\",\\\"name\\\":\\\"installation-pull-secrets\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/openshift/serviceaccount\\\",\\\"name\\\":\\\"bound-sa-token\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ntfs8\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-image-registry\"/\"image-registry-6cfdfbdc99-nltls\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-image-registry/image-registry-6cfdfbdc99-nltls" Sep 07 17:10:52.472384 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:52.472332 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34314299-5ca7-46ec-9159-8568af291e23\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-07T17:10:37Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"},\\\"containerID\\\":\\\"cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"console\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"100Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T17:10:27Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000210000],\\\"uid\\\":1000210000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/serving-cert\\\",\\\"name\\\":\\\"console-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-config\\\",\\\"name\\\":\\\"console-oauth-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/console-config\\\",\\\"name\\\":\\\"console-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/service-ca\\\",\\\"name\\\":\\\"service-ca\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/pki/ca-trust/extracted/pem\\\",\\\"name\\\":\\\"trusted-ca-bundle\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/oauth-serving-cert\\\",\\\"name\\\":\\\"oauth-serving-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-nw9wr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-console\"/\"console-7d6886fd7f-dp44v\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/pod?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:10:55.898086 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:55.897929 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4ff4684e81eda96da31b31effc7526407ab4240ab95dbd354e9d6cf7c080d1a5\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\"],\\\"sizeBytes\\\":1191140208},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d70ac620aaa7a359b6f946c7bbc7c43ec8f662e3dbc01a81bd3b874a50d8ceda\\\"],\\\"sizeBytes\\\":1187853995},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3863ee2168370cd55c84fc19f023ca4f9af4ba51c20443d7a6736e4099040ec6\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7a667dfb8d993459f4da0daac5ac73a74d860d523a74dbe9fe585e023862ba49\\\"],\\\"sizeBytes\\\":880865337},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4df5726479a4318422f562945c7cea7d73050af86def291ece6bcd0d5f671334\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399\\\"],\\\"sizeBytes\\\":591237733},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:526a2ac4ac8a3b16cc43e71f40cd285f79666513fb48205338da74737b4cfc31\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:feb1137f1c622b2ed3fd7864ae97b30c49ca40be335e968e733ae1387f619486\\\"],\\\"sizeBytes\\\":585209957},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9c68cfdf784793c266ed7fae433db183b4521a7bf34f1a63d7c2c410da6c05d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e65a8a319bd1e96e1eae496e0eb389dd5cabc8fbc7bab1f92eb74728b74ecf27\\\"],\\\"sizeBytes\\\":580711434},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1c6b55e6f6dc3f07ab2626923169873734c7376bf8660d673a127e97047ed6ce\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:34db7d14a3db6337de33719bd3025cda31d8b37717af24b6037967f43a8739f4\\\"],\\\"sizeBytes\\\":542930692},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:399e68fa23665c69938366fdc04639a184686afe6786763a9426fef843148e82\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:602e04dc7e5aa127c231a39d35a6271ada557045ff0ce999454898cfa7a0e88c\\\"],\\\"sizeBytes\\\":460953253},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9b492b260407a9e94419d1ba959aee98addc71f43727d884bf384001dd8e90a\\\"],\\\"sizeBytes\\\":443008130},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31f886d680e8409c2a25409445b0b1c2a87d18f297a7b0ed01216d07e36d6768\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f97cdd83d19e4e52a5bb1f4767a09012d2e6ad45c78f64fabd0aa9bfda98c439\\\"],\\\"sizeBytes\\\":391508249},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2715231eb5735496339b35b0019240e8ed476f49005c78a9b0282fff549b316e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:442f6f711db8a6f8fae4dfff48a29e5fb0d154b2c6779c5e2e46d649f99ccfec\\\"],\\\"sizeBytes\\\":353954431},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:40bc1b9c6b608369978e9f165c5e22d7a75fd91b3480cba60cd275228d721616\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b791ac4a74fb40f5560b4ee0887bab8d3cd5115c5423349724fd65cdbf525d8\\\"],\\\"sizeBytes\\\":351323595},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d2fae617617ccbdef514e3068eb3db7ed6f5869fe748bbfc9d66eb536d3e26f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1b0c1ab26ecd2490e7084728c14cee5298a3d8697acc8d2c75021a2f7ad1f75\\\"],\\\"sizeBytes\\\":348544277},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1f7586513a5097635faee33f4146564c1b0d4edd23c21c2521becba0d402f955\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e41b81f826f48a37a241bb4e5afd81263359bd8ac45c7fcec4501ce841546cb7\\\"],\\\"sizeBytes\\\":342478082},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1483a45f7f5e30c4dc46b45edcbb4b3f83cb20153c8153e2663989ee0fcd4e1e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f07372c9659382347c892a0489ee20746fd78a745fc570697fed5635ac94c20\\\"],\\\"sizeBytes\\\":322850422},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ca84fc8b10f7856b22f27b4ad37d3f03f9bed9c51904a00e640f21ea99d49487\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\\\"],\\\"sizeBytes\\\":321671430},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\"],\\\"sizeBytes\\\":317668265},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b4ab7e29d1a1eff8fee85968132d3d721e53af925fe6b2971a3a4527a9bee53e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db7298018960838aec34d162c60cc7865f4ab2a95dcf0db50dbc8f835c00f0d6\\\"],\\\"sizeBytes\\\":293038040},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9018c47455af82aa9aba488b18f1fb9824a505ad46539494602fe936ffc4b601\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f24daee8722c23f953d9eb0f79bd54791c87e5c3b5c46eb5d9f04aeabe6c7c5f\\\"],\\\"sizeBytes\\\":288598319},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:095960e9b22d8660a23c0cdcd3a7961988ea355e189f9faf9008a7e2a0824286\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924fcb9b7fe53e9cdd1b799ec020159588f018457a2c9f24a3aa4d960c34ccbf\\\"],\\\"sizeBytes\\\":286680505},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2afe1e650f7a2bc2eb7ce312884b1269e46e09583911137ab152cedc3a022511\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\"],\\\"sizeBytes\\\":282837706},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:729748ec5850dc51c711ce454f83b0bf3e943920c806fde555bbf80cde3af394\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e813bebbf87427d9525d9fc7af34884643454e06070fdb1c4c13e0d7c4a49e54\\\"],\\\"sizeBytes\\\":280950699},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:41872f8528a5e9d52c4f1acc02682c0f5ebaca4a6ae3c2b175afa117e2b52b9f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c503b5dafe843398d33e21a2192b2aac1140ae4fff443362a8a8e30b7ba591a3\\\"],\\\"sizeBytes\\\":274917752},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5d80239da456006cca8562483fc656256da631e5bd53bdb148cf944fff79f699\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9ccee41603155e1ae29ca89df28ec1c12dd93e7d46098d032ecec1e8d8215b6e\\\"],\\\"sizeBytes\\\":272736506},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1d17bd2248667a524fcaafa91cdb351cad919be880335d6eb04ee24c658928c2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:90d2319a9a2c6beea025f4be564f5ce39082fd4d7e74dada5b3012e790942555\\\"],\\\"sizeBytes\\\":255512859},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d8ec578bbd6716e84b3f12ae63e637d79aec1b16fb03556784b8ed5fce16d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6cb6bdafd8554a7b8765137e45621fe33091c54b98ab962ff24e5f8c3fd2c531\\\"],\\\"sizeBytes\\\":254830869},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35d1ee61dada1d62dcc613c552ad9918bd7d2a6132a081d53de4775d3d16841c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6beed8f25b71249945ab6789b94880b9c061dc53d3bb9f4cad17a5ca9de88e29\\\"],\\\"sizeBytes\\\":253435945},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:015b604a274fa1896122e0e4690585b3a83549f7c8670931a798446eec8963d0\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c487357748145bddf8d3d1f7036542da604a17f08f73602612838579d1bd3ac\\\"],\\\"sizeBytes\\\":252846435},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076da7d59107bba89a1430a080176af4f83052296230df56d51b724cf03a169e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e3a2f3c9f8d7a459ccd0719643fba2d050d158d387ade828c3d58813bc3b6e4\\\"],\\\"sizeBytes\\\":240112805},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:909d94acdf16bd26857849698608cb17db0c62ca3eab7d670ee858a6e839390e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ead92f00c83a562232b8feac34089875e63ff76080039dc4c09e8ed9cc22d5fb\\\"],\\\"sizeBytes\\\":239854772},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b7540027380dafacdaf56cac1129e87b9081a45e3906f9363eb19d5a57ecf12\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e53129eef13d9deaac86d2e0f99a320d5b7b28d8347c5418169db98a679913de\\\"],\\\"sizeBytes\\\":229649173},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dba6a8cc4601d0c588a8dde112534f8e60a458bec75034ab9ed7dac6d45d0875\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e35f17762fc3f961f361749a5e9d544b17011c64187f85e085c763204f33a113\\\"],\\\"sizeBytes\\\":227367336},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2ed6cd765c62307186d486139e35584ce657840609d2147d464cae3481404155\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee7e710dffa2957f4ef74529a60af22f22f5e7846bcac2f8e8743cb57628b322\\\"],\\\"sizeBytes\\\":222920470},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fb1140d1be7f58dafc56142241b6733f8b60b960b64dcb57585ef17d9cc459b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f675d5827e7bef4552ea615e94b398c323676551997d51bec2e58778e962657\\\"],\\\"sizeBytes\\\":220121222},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bfe8897012a86426822069e3423480528ca16f1610a60eefd6bc70e0e140d941\\\"],\\\"sizeBytes\\\":210996836}]}}\" for node \"ip-10-0-143-85.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/node?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" Sep 07 17:10:55.918630 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:55.918512 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4ff4684e81eda96da31b31effc7526407ab4240ab95dbd354e9d6cf7c080d1a5\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\"],\\\"sizeBytes\\\":1191140208},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d70ac620aaa7a359b6f946c7bbc7c43ec8f662e3dbc01a81bd3b874a50d8ceda\\\"],\\\"sizeBytes\\\":1187853995},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3863ee2168370cd55c84fc19f023ca4f9af4ba51c20443d7a6736e4099040ec6\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7a667dfb8d993459f4da0daac5ac73a74d860d523a74dbe9fe585e023862ba49\\\"],\\\"sizeBytes\\\":880865337},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4df5726479a4318422f562945c7cea7d73050af86def291ece6bcd0d5f671334\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399\\\"],\\\"sizeBytes\\\":591237733},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:526a2ac4ac8a3b16cc43e71f40cd285f79666513fb48205338da74737b4cfc31\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:feb1137f1c622b2ed3fd7864ae97b30c49ca40be335e968e733ae1387f619486\\\"],\\\"sizeBytes\\\":585209957},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9c68cfdf784793c266ed7fae433db183b4521a7bf34f1a63d7c2c410da6c05d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e65a8a319bd1e96e1eae496e0eb389dd5cabc8fbc7bab1f92eb74728b74ecf27\\\"],\\\"sizeBytes\\\":580711434},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1c6b55e6f6dc3f07ab2626923169873734c7376bf8660d673a127e97047ed6ce\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:34db7d14a3db6337de33719bd3025cda31d8b37717af24b6037967f43a8739f4\\\"],\\\"sizeBytes\\\":542930692},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:399e68fa23665c69938366fdc04639a184686afe6786763a9426fef843148e82\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:602e04dc7e5aa127c231a39d35a6271ada557045ff0ce999454898cfa7a0e88c\\\"],\\\"sizeBytes\\\":460953253},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9b492b260407a9e94419d1ba959aee98addc71f43727d884bf384001dd8e90a\\\"],\\\"sizeBytes\\\":443008130},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31f886d680e8409c2a25409445b0b1c2a87d18f297a7b0ed01216d07e36d6768\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f97cdd83d19e4e52a5bb1f4767a09012d2e6ad45c78f64fabd0aa9bfda98c439\\\"],\\\"sizeBytes\\\":391508249},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2715231eb5735496339b35b0019240e8ed476f49005c78a9b0282fff549b316e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:442f6f711db8a6f8fae4dfff48a29e5fb0d154b2c6779c5e2e46d649f99ccfec\\\"],\\\"sizeBytes\\\":353954431},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:40bc1b9c6b608369978e9f165c5e22d7a75fd91b3480cba60cd275228d721616\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b791ac4a74fb40f5560b4ee0887bab8d3cd5115c5423349724fd65cdbf525d8\\\"],\\\"sizeBytes\\\":351323595},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d2fae617617ccbdef514e3068eb3db7ed6f5869fe748bbfc9d66eb536d3e26f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1b0c1ab26ecd2490e7084728c14cee5298a3d8697acc8d2c75021a2f7ad1f75\\\"],\\\"sizeBytes\\\":348544277},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1f7586513a5097635faee33f4146564c1b0d4edd23c21c2521becba0d402f955\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e41b81f826f48a37a241bb4e5afd81263359bd8ac45c7fcec4501ce841546cb7\\\"],\\\"sizeBytes\\\":342478082},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1483a45f7f5e30c4dc46b45edcbb4b3f83cb20153c8153e2663989ee0fcd4e1e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f07372c9659382347c892a0489ee20746fd78a745fc570697fed5635ac94c20\\\"],\\\"sizeBytes\\\":322850422},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ca84fc8b10f7856b22f27b4ad37d3f03f9bed9c51904a00e640f21ea99d49487\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\\\"],\\\"sizeBytes\\\":321671430},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\"],\\\"sizeBytes\\\":317668265},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b4ab7e29d1a1eff8fee85968132d3d721e53af925fe6b2971a3a4527a9bee53e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db7298018960838aec34d162c60cc7865f4ab2a95dcf0db50dbc8f835c00f0d6\\\"],\\\"sizeBytes\\\":293038040},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9018c47455af82aa9aba488b18f1fb9824a505ad46539494602fe936ffc4b601\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f24daee8722c23f953d9eb0f79bd54791c87e5c3b5c46eb5d9f04aeabe6c7c5f\\\"],\\\"sizeBytes\\\":288598319},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:095960e9b22d8660a23c0cdcd3a7961988ea355e189f9faf9008a7e2a0824286\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924fcb9b7fe53e9cdd1b799ec020159588f018457a2c9f24a3aa4d960c34ccbf\\\"],\\\"sizeBytes\\\":286680505},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2afe1e650f7a2bc2eb7ce312884b1269e46e09583911137ab152cedc3a022511\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\"],\\\"sizeBytes\\\":282837706},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:729748ec5850dc51c711ce454f83b0bf3e943920c806fde555bbf80cde3af394\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e813bebbf87427d9525d9fc7af34884643454e06070fdb1c4c13e0d7c4a49e54\\\"],\\\"sizeBytes\\\":280950699},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:41872f8528a5e9d52c4f1acc02682c0f5ebaca4a6ae3c2b175afa117e2b52b9f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c503b5dafe843398d33e21a2192b2aac1140ae4fff443362a8a8e30b7ba591a3\\\"],\\\"sizeBytes\\\":274917752},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5d80239da456006cca8562483fc656256da631e5bd53bdb148cf944fff79f699\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9ccee41603155e1ae29ca89df28ec1c12dd93e7d46098d032ecec1e8d8215b6e\\\"],\\\"sizeBytes\\\":272736506},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1d17bd2248667a524fcaafa91cdb351cad919be880335d6eb04ee24c658928c2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:90d2319a9a2c6beea025f4be564f5ce39082fd4d7e74dada5b3012e790942555\\\"],\\\"sizeBytes\\\":255512859},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d8ec578bbd6716e84b3f12ae63e637d79aec1b16fb03556784b8ed5fce16d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6cb6bdafd8554a7b8765137e45621fe33091c54b98ab962ff24e5f8c3fd2c531\\\"],\\\"sizeBytes\\\":254830869},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35d1ee61dada1d62dcc613c552ad9918bd7d2a6132a081d53de4775d3d16841c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6beed8f25b71249945ab6789b94880b9c061dc53d3bb9f4cad17a5ca9de88e29\\\"],\\\"sizeBytes\\\":253435945},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:015b604a274fa1896122e0e4690585b3a83549f7c8670931a798446eec8963d0\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c487357748145bddf8d3d1f7036542da604a17f08f73602612838579d1bd3ac\\\"],\\\"sizeBytes\\\":252846435},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076da7d59107bba89a1430a080176af4f83052296230df56d51b724cf03a169e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e3a2f3c9f8d7a459ccd0719643fba2d050d158d387ade828c3d58813bc3b6e4\\\"],\\\"sizeBytes\\\":240112805},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:909d94acdf16bd26857849698608cb17db0c62ca3eab7d670ee858a6e839390e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ead92f00c83a562232b8feac34089875e63ff76080039dc4c09e8ed9cc22d5fb\\\"],\\\"sizeBytes\\\":239854772},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b7540027380dafacdaf56cac1129e87b9081a45e3906f9363eb19d5a57ecf12\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e53129eef13d9deaac86d2e0f99a320d5b7b28d8347c5418169db98a679913de\\\"],\\\"sizeBytes\\\":229649173},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dba6a8cc4601d0c588a8dde112534f8e60a458bec75034ab9ed7dac6d45d0875\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e35f17762fc3f961f361749a5e9d544b17011c64187f85e085c763204f33a113\\\"],\\\"sizeBytes\\\":227367336},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2ed6cd765c62307186d486139e35584ce657840609d2147d464cae3481404155\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee7e710dffa2957f4ef74529a60af22f22f5e7846bcac2f8e8743cb57628b322\\\"],\\\"sizeBytes\\\":222920470},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fb1140d1be7f58dafc56142241b6733f8b60b960b64dcb57585ef17d9cc459b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f675d5827e7bef4552ea615e94b398c323676551997d51bec2e58778e962657\\\"],\\\"sizeBytes\\\":220121222},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bfe8897012a86426822069e3423480528ca16f1610a60eefd6bc70e0e140d941\\\"],\\\"sizeBytes\\\":210996836}]}}\" for node \"ip-10-0-143-85.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/node?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" Sep 07 17:10:55.949327 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:55.949208 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4ff4684e81eda96da31b31effc7526407ab4240ab95dbd354e9d6cf7c080d1a5\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\"],\\\"sizeBytes\\\":1191140208},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d70ac620aaa7a359b6f946c7bbc7c43ec8f662e3dbc01a81bd3b874a50d8ceda\\\"],\\\"sizeBytes\\\":1187853995},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3863ee2168370cd55c84fc19f023ca4f9af4ba51c20443d7a6736e4099040ec6\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7a667dfb8d993459f4da0daac5ac73a74d860d523a74dbe9fe585e023862ba49\\\"],\\\"sizeBytes\\\":880865337},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4df5726479a4318422f562945c7cea7d73050af86def291ece6bcd0d5f671334\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399\\\"],\\\"sizeBytes\\\":591237733},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:526a2ac4ac8a3b16cc43e71f40cd285f79666513fb48205338da74737b4cfc31\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:feb1137f1c622b2ed3fd7864ae97b30c49ca40be335e968e733ae1387f619486\\\"],\\\"sizeBytes\\\":585209957},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9c68cfdf784793c266ed7fae433db183b4521a7bf34f1a63d7c2c410da6c05d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e65a8a319bd1e96e1eae496e0eb389dd5cabc8fbc7bab1f92eb74728b74ecf27\\\"],\\\"sizeBytes\\\":580711434},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1c6b55e6f6dc3f07ab2626923169873734c7376bf8660d673a127e97047ed6ce\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:34db7d14a3db6337de33719bd3025cda31d8b37717af24b6037967f43a8739f4\\\"],\\\"sizeBytes\\\":542930692},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:399e68fa23665c69938366fdc04639a184686afe6786763a9426fef843148e82\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:602e04dc7e5aa127c231a39d35a6271ada557045ff0ce999454898cfa7a0e88c\\\"],\\\"sizeBytes\\\":460953253},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9b492b260407a9e94419d1ba959aee98addc71f43727d884bf384001dd8e90a\\\"],\\\"sizeBytes\\\":443008130},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31f886d680e8409c2a25409445b0b1c2a87d18f297a7b0ed01216d07e36d6768\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f97cdd83d19e4e52a5bb1f4767a09012d2e6ad45c78f64fabd0aa9bfda98c439\\\"],\\\"sizeBytes\\\":391508249},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2715231eb5735496339b35b0019240e8ed476f49005c78a9b0282fff549b316e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:442f6f711db8a6f8fae4dfff48a29e5fb0d154b2c6779c5e2e46d649f99ccfec\\\"],\\\"sizeBytes\\\":353954431},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:40bc1b9c6b608369978e9f165c5e22d7a75fd91b3480cba60cd275228d721616\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b791ac4a74fb40f5560b4ee0887bab8d3cd5115c5423349724fd65cdbf525d8\\\"],\\\"sizeBytes\\\":351323595},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d2fae617617ccbdef514e3068eb3db7ed6f5869fe748bbfc9d66eb536d3e26f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1b0c1ab26ecd2490e7084728c14cee5298a3d8697acc8d2c75021a2f7ad1f75\\\"],\\\"sizeBytes\\\":348544277},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1f7586513a5097635faee33f4146564c1b0d4edd23c21c2521becba0d402f955\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e41b81f826f48a37a241bb4e5afd81263359bd8ac45c7fcec4501ce841546cb7\\\"],\\\"sizeBytes\\\":342478082},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1483a45f7f5e30c4dc46b45edcbb4b3f83cb20153c8153e2663989ee0fcd4e1e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f07372c9659382347c892a0489ee20746fd78a745fc570697fed5635ac94c20\\\"],\\\"sizeBytes\\\":322850422},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ca84fc8b10f7856b22f27b4ad37d3f03f9bed9c51904a00e640f21ea99d49487\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\\\"],\\\"sizeBytes\\\":321671430},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\"],\\\"sizeBytes\\\":317668265},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b4ab7e29d1a1eff8fee85968132d3d721e53af925fe6b2971a3a4527a9bee53e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db7298018960838aec34d162c60cc7865f4ab2a95dcf0db50dbc8f835c00f0d6\\\"],\\\"sizeBytes\\\":293038040},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9018c47455af82aa9aba488b18f1fb9824a505ad46539494602fe936ffc4b601\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f24daee8722c23f953d9eb0f79bd54791c87e5c3b5c46eb5d9f04aeabe6c7c5f\\\"],\\\"sizeBytes\\\":288598319},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:095960e9b22d8660a23c0cdcd3a7961988ea355e189f9faf9008a7e2a0824286\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924fcb9b7fe53e9cdd1b799ec020159588f018457a2c9f24a3aa4d960c34ccbf\\\"],\\\"sizeBytes\\\":286680505},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2afe1e650f7a2bc2eb7ce312884b1269e46e09583911137ab152cedc3a022511\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\"],\\\"sizeBytes\\\":282837706},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:729748ec5850dc51c711ce454f83b0bf3e943920c806fde555bbf80cde3af394\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e813bebbf87427d9525d9fc7af34884643454e06070fdb1c4c13e0d7c4a49e54\\\"],\\\"sizeBytes\\\":280950699},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:41872f8528a5e9d52c4f1acc02682c0f5ebaca4a6ae3c2b175afa117e2b52b9f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c503b5dafe843398d33e21a2192b2aac1140ae4fff443362a8a8e30b7ba591a3\\\"],\\\"sizeBytes\\\":274917752},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5d80239da456006cca8562483fc656256da631e5bd53bdb148cf944fff79f699\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9ccee41603155e1ae29ca89df28ec1c12dd93e7d46098d032ecec1e8d8215b6e\\\"],\\\"sizeBytes\\\":272736506},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1d17bd2248667a524fcaafa91cdb351cad919be880335d6eb04ee24c658928c2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:90d2319a9a2c6beea025f4be564f5ce39082fd4d7e74dada5b3012e790942555\\\"],\\\"sizeBytes\\\":255512859},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d8ec578bbd6716e84b3f12ae63e637d79aec1b16fb03556784b8ed5fce16d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6cb6bdafd8554a7b8765137e45621fe33091c54b98ab962ff24e5f8c3fd2c531\\\"],\\\"sizeBytes\\\":254830869},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35d1ee61dada1d62dcc613c552ad9918bd7d2a6132a081d53de4775d3d16841c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6beed8f25b71249945ab6789b94880b9c061dc53d3bb9f4cad17a5ca9de88e29\\\"],\\\"sizeBytes\\\":253435945},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:015b604a274fa1896122e0e4690585b3a83549f7c8670931a798446eec8963d0\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c487357748145bddf8d3d1f7036542da604a17f08f73602612838579d1bd3ac\\\"],\\\"sizeBytes\\\":252846435},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076da7d59107bba89a1430a080176af4f83052296230df56d51b724cf03a169e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e3a2f3c9f8d7a459ccd0719643fba2d050d158d387ade828c3d58813bc3b6e4\\\"],\\\"sizeBytes\\\":240112805},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:909d94acdf16bd26857849698608cb17db0c62ca3eab7d670ee858a6e839390e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ead92f00c83a562232b8feac34089875e63ff76080039dc4c09e8ed9cc22d5fb\\\"],\\\"sizeBytes\\\":239854772},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b7540027380dafacdaf56cac1129e87b9081a45e3906f9363eb19d5a57ecf12\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e53129eef13d9deaac86d2e0f99a320d5b7b28d8347c5418169db98a679913de\\\"],\\\"sizeBytes\\\":229649173},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dba6a8cc4601d0c588a8dde112534f8e60a458bec75034ab9ed7dac6d45d0875\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e35f17762fc3f961f361749a5e9d544b17011c64187f85e085c763204f33a113\\\"],\\\"sizeBytes\\\":227367336},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2ed6cd765c62307186d486139e35584ce657840609d2147d464cae3481404155\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee7e710dffa2957f4ef74529a60af22f22f5e7846bcac2f8e8743cb57628b322\\\"],\\\"sizeBytes\\\":222920470},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fb1140d1be7f58dafc56142241b6733f8b60b960b64dcb57585ef17d9cc459b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f675d5827e7bef4552ea615e94b398c323676551997d51bec2e58778e962657\\\"],\\\"sizeBytes\\\":220121222},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bfe8897012a86426822069e3423480528ca16f1610a60eefd6bc70e0e140d941\\\"],\\\"sizeBytes\\\":210996836}]}}\" for node \"ip-10-0-143-85.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/node?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" Sep 07 17:10:55.970540 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:55.970424 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4ff4684e81eda96da31b31effc7526407ab4240ab95dbd354e9d6cf7c080d1a5\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\"],\\\"sizeBytes\\\":1191140208},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d70ac620aaa7a359b6f946c7bbc7c43ec8f662e3dbc01a81bd3b874a50d8ceda\\\"],\\\"sizeBytes\\\":1187853995},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3863ee2168370cd55c84fc19f023ca4f9af4ba51c20443d7a6736e4099040ec6\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7a667dfb8d993459f4da0daac5ac73a74d860d523a74dbe9fe585e023862ba49\\\"],\\\"sizeBytes\\\":880865337},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4df5726479a4318422f562945c7cea7d73050af86def291ece6bcd0d5f671334\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399\\\"],\\\"sizeBytes\\\":591237733},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:526a2ac4ac8a3b16cc43e71f40cd285f79666513fb48205338da74737b4cfc31\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:feb1137f1c622b2ed3fd7864ae97b30c49ca40be335e968e733ae1387f619486\\\"],\\\"sizeBytes\\\":585209957},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9c68cfdf784793c266ed7fae433db183b4521a7bf34f1a63d7c2c410da6c05d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e65a8a319bd1e96e1eae496e0eb389dd5cabc8fbc7bab1f92eb74728b74ecf27\\\"],\\\"sizeBytes\\\":580711434},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1c6b55e6f6dc3f07ab2626923169873734c7376bf8660d673a127e97047ed6ce\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:34db7d14a3db6337de33719bd3025cda31d8b37717af24b6037967f43a8739f4\\\"],\\\"sizeBytes\\\":542930692},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:399e68fa23665c69938366fdc04639a184686afe6786763a9426fef843148e82\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:602e04dc7e5aa127c231a39d35a6271ada557045ff0ce999454898cfa7a0e88c\\\"],\\\"sizeBytes\\\":460953253},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9b492b260407a9e94419d1ba959aee98addc71f43727d884bf384001dd8e90a\\\"],\\\"sizeBytes\\\":443008130},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31f886d680e8409c2a25409445b0b1c2a87d18f297a7b0ed01216d07e36d6768\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f97cdd83d19e4e52a5bb1f4767a09012d2e6ad45c78f64fabd0aa9bfda98c439\\\"],\\\"sizeBytes\\\":391508249},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2715231eb5735496339b35b0019240e8ed476f49005c78a9b0282fff549b316e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:442f6f711db8a6f8fae4dfff48a29e5fb0d154b2c6779c5e2e46d649f99ccfec\\\"],\\\"sizeBytes\\\":353954431},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:40bc1b9c6b608369978e9f165c5e22d7a75fd91b3480cba60cd275228d721616\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b791ac4a74fb40f5560b4ee0887bab8d3cd5115c5423349724fd65cdbf525d8\\\"],\\\"sizeBytes\\\":351323595},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d2fae617617ccbdef514e3068eb3db7ed6f5869fe748bbfc9d66eb536d3e26f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1b0c1ab26ecd2490e7084728c14cee5298a3d8697acc8d2c75021a2f7ad1f75\\\"],\\\"sizeBytes\\\":348544277},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1f7586513a5097635faee33f4146564c1b0d4edd23c21c2521becba0d402f955\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e41b81f826f48a37a241bb4e5afd81263359bd8ac45c7fcec4501ce841546cb7\\\"],\\\"sizeBytes\\\":342478082},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1483a45f7f5e30c4dc46b45edcbb4b3f83cb20153c8153e2663989ee0fcd4e1e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f07372c9659382347c892a0489ee20746fd78a745fc570697fed5635ac94c20\\\"],\\\"sizeBytes\\\":322850422},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ca84fc8b10f7856b22f27b4ad37d3f03f9bed9c51904a00e640f21ea99d49487\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\\\"],\\\"sizeBytes\\\":321671430},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\"],\\\"sizeBytes\\\":317668265},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b4ab7e29d1a1eff8fee85968132d3d721e53af925fe6b2971a3a4527a9bee53e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db7298018960838aec34d162c60cc7865f4ab2a95dcf0db50dbc8f835c00f0d6\\\"],\\\"sizeBytes\\\":293038040},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9018c47455af82aa9aba488b18f1fb9824a505ad46539494602fe936ffc4b601\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f24daee8722c23f953d9eb0f79bd54791c87e5c3b5c46eb5d9f04aeabe6c7c5f\\\"],\\\"sizeBytes\\\":288598319},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:095960e9b22d8660a23c0cdcd3a7961988ea355e189f9faf9008a7e2a0824286\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924fcb9b7fe53e9cdd1b799ec020159588f018457a2c9f24a3aa4d960c34ccbf\\\"],\\\"sizeBytes\\\":286680505},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2afe1e650f7a2bc2eb7ce312884b1269e46e09583911137ab152cedc3a022511\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\"],\\\"sizeBytes\\\":282837706},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:729748ec5850dc51c711ce454f83b0bf3e943920c806fde555bbf80cde3af394\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e813bebbf87427d9525d9fc7af34884643454e06070fdb1c4c13e0d7c4a49e54\\\"],\\\"sizeBytes\\\":280950699},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:41872f8528a5e9d52c4f1acc02682c0f5ebaca4a6ae3c2b175afa117e2b52b9f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c503b5dafe843398d33e21a2192b2aac1140ae4fff443362a8a8e30b7ba591a3\\\"],\\\"sizeBytes\\\":274917752},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5d80239da456006cca8562483fc656256da631e5bd53bdb148cf944fff79f699\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9ccee41603155e1ae29ca89df28ec1c12dd93e7d46098d032ecec1e8d8215b6e\\\"],\\\"sizeBytes\\\":272736506},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1d17bd2248667a524fcaafa91cdb351cad919be880335d6eb04ee24c658928c2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:90d2319a9a2c6beea025f4be564f5ce39082fd4d7e74dada5b3012e790942555\\\"],\\\"sizeBytes\\\":255512859},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d8ec578bbd6716e84b3f12ae63e637d79aec1b16fb03556784b8ed5fce16d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6cb6bdafd8554a7b8765137e45621fe33091c54b98ab962ff24e5f8c3fd2c531\\\"],\\\"sizeBytes\\\":254830869},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35d1ee61dada1d62dcc613c552ad9918bd7d2a6132a081d53de4775d3d16841c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6beed8f25b71249945ab6789b94880b9c061dc53d3bb9f4cad17a5ca9de88e29\\\"],\\\"sizeBytes\\\":253435945},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:015b604a274fa1896122e0e4690585b3a83549f7c8670931a798446eec8963d0\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c487357748145bddf8d3d1f7036542da604a17f08f73602612838579d1bd3ac\\\"],\\\"sizeBytes\\\":252846435},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076da7d59107bba89a1430a080176af4f83052296230df56d51b724cf03a169e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e3a2f3c9f8d7a459ccd0719643fba2d050d158d387ade828c3d58813bc3b6e4\\\"],\\\"sizeBytes\\\":240112805},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:909d94acdf16bd26857849698608cb17db0c62ca3eab7d670ee858a6e839390e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ead92f00c83a562232b8feac34089875e63ff76080039dc4c09e8ed9cc22d5fb\\\"],\\\"sizeBytes\\\":239854772},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b7540027380dafacdaf56cac1129e87b9081a45e3906f9363eb19d5a57ecf12\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e53129eef13d9deaac86d2e0f99a320d5b7b28d8347c5418169db98a679913de\\\"],\\\"sizeBytes\\\":229649173},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dba6a8cc4601d0c588a8dde112534f8e60a458bec75034ab9ed7dac6d45d0875\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e35f17762fc3f961f361749a5e9d544b17011c64187f85e085c763204f33a113\\\"],\\\"sizeBytes\\\":227367336},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2ed6cd765c62307186d486139e35584ce657840609d2147d464cae3481404155\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee7e710dffa2957f4ef74529a60af22f22f5e7846bcac2f8e8743cb57628b322\\\"],\\\"sizeBytes\\\":222920470},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fb1140d1be7f58dafc56142241b6733f8b60b960b64dcb57585ef17d9cc459b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f675d5827e7bef4552ea615e94b398c323676551997d51bec2e58778e962657\\\"],\\\"sizeBytes\\\":220121222},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bfe8897012a86426822069e3423480528ca16f1610a60eefd6bc70e0e140d941\\\"],\\\"sizeBytes\\\":210996836}]}}\" for node \"ip-10-0-143-85.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/node?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" Sep 07 17:10:56.000487 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:56.000368 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T17:10:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4ff4684e81eda96da31b31effc7526407ab4240ab95dbd354e9d6cf7c080d1a5\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\"],\\\"sizeBytes\\\":1191140208},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d70ac620aaa7a359b6f946c7bbc7c43ec8f662e3dbc01a81bd3b874a50d8ceda\\\"],\\\"sizeBytes\\\":1187853995},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3863ee2168370cd55c84fc19f023ca4f9af4ba51c20443d7a6736e4099040ec6\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7a667dfb8d993459f4da0daac5ac73a74d860d523a74dbe9fe585e023862ba49\\\"],\\\"sizeBytes\\\":880865337},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4df5726479a4318422f562945c7cea7d73050af86def291ece6bcd0d5f671334\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399\\\"],\\\"sizeBytes\\\":591237733},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:526a2ac4ac8a3b16cc43e71f40cd285f79666513fb48205338da74737b4cfc31\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:feb1137f1c622b2ed3fd7864ae97b30c49ca40be335e968e733ae1387f619486\\\"],\\\"sizeBytes\\\":585209957},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9c68cfdf784793c266ed7fae433db183b4521a7bf34f1a63d7c2c410da6c05d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e65a8a319bd1e96e1eae496e0eb389dd5cabc8fbc7bab1f92eb74728b74ecf27\\\"],\\\"sizeBytes\\\":580711434},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1c6b55e6f6dc3f07ab2626923169873734c7376bf8660d673a127e97047ed6ce\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:34db7d14a3db6337de33719bd3025cda31d8b37717af24b6037967f43a8739f4\\\"],\\\"sizeBytes\\\":542930692},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:399e68fa23665c69938366fdc04639a184686afe6786763a9426fef843148e82\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:602e04dc7e5aa127c231a39d35a6271ada557045ff0ce999454898cfa7a0e88c\\\"],\\\"sizeBytes\\\":460953253},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d6a9d94f5df316cf2ebf6bed9a19dbf1bf506dae0e2f15792c1dcce818e2146\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9b492b260407a9e94419d1ba959aee98addc71f43727d884bf384001dd8e90a\\\"],\\\"sizeBytes\\\":443008130},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:31f886d680e8409c2a25409445b0b1c2a87d18f297a7b0ed01216d07e36d6768\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f97cdd83d19e4e52a5bb1f4767a09012d2e6ad45c78f64fabd0aa9bfda98c439\\\"],\\\"sizeBytes\\\":391508249},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2715231eb5735496339b35b0019240e8ed476f49005c78a9b0282fff549b316e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:442f6f711db8a6f8fae4dfff48a29e5fb0d154b2c6779c5e2e46d649f99ccfec\\\"],\\\"sizeBytes\\\":353954431},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:40bc1b9c6b608369978e9f165c5e22d7a75fd91b3480cba60cd275228d721616\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b791ac4a74fb40f5560b4ee0887bab8d3cd5115c5423349724fd65cdbf525d8\\\"],\\\"sizeBytes\\\":351323595},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d2fae617617ccbdef514e3068eb3db7ed6f5869fe748bbfc9d66eb536d3e26f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1b0c1ab26ecd2490e7084728c14cee5298a3d8697acc8d2c75021a2f7ad1f75\\\"],\\\"sizeBytes\\\":348544277},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1f7586513a5097635faee33f4146564c1b0d4edd23c21c2521becba0d402f955\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e41b81f826f48a37a241bb4e5afd81263359bd8ac45c7fcec4501ce841546cb7\\\"],\\\"sizeBytes\\\":342478082},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1483a45f7f5e30c4dc46b45edcbb4b3f83cb20153c8153e2663989ee0fcd4e1e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f07372c9659382347c892a0489ee20746fd78a745fc570697fed5635ac94c20\\\"],\\\"sizeBytes\\\":322850422},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ca84fc8b10f7856b22f27b4ad37d3f03f9bed9c51904a00e640f21ea99d49487\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\\\"],\\\"sizeBytes\\\":321671430},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:069a7c6581b9d90c26f110eba7bd568c1a63bc4c0c3101b806ee9619b5eff81e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dcaa1d46224a73ba469d38c7a644f6cf5a1276f0b38c78d6bee6785177ec7162\\\"],\\\"sizeBytes\\\":317668265},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b4ab7e29d1a1eff8fee85968132d3d721e53af925fe6b2971a3a4527a9bee53e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db7298018960838aec34d162c60cc7865f4ab2a95dcf0db50dbc8f835c00f0d6\\\"],\\\"sizeBytes\\\":293038040},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9018c47455af82aa9aba488b18f1fb9824a505ad46539494602fe936ffc4b601\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f24daee8722c23f953d9eb0f79bd54791c87e5c3b5c46eb5d9f04aeabe6c7c5f\\\"],\\\"sizeBytes\\\":288598319},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:095960e9b22d8660a23c0cdcd3a7961988ea355e189f9faf9008a7e2a0824286\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924fcb9b7fe53e9cdd1b799ec020159588f018457a2c9f24a3aa4d960c34ccbf\\\"],\\\"sizeBytes\\\":286680505},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2afe1e650f7a2bc2eb7ce312884b1269e46e09583911137ab152cedc3a022511\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\"],\\\"sizeBytes\\\":282837706},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:729748ec5850dc51c711ce454f83b0bf3e943920c806fde555bbf80cde3af394\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e813bebbf87427d9525d9fc7af34884643454e06070fdb1c4c13e0d7c4a49e54\\\"],\\\"sizeBytes\\\":280950699},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:41872f8528a5e9d52c4f1acc02682c0f5ebaca4a6ae3c2b175afa117e2b52b9f\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c503b5dafe843398d33e21a2192b2aac1140ae4fff443362a8a8e30b7ba591a3\\\"],\\\"sizeBytes\\\":274917752},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5d80239da456006cca8562483fc656256da631e5bd53bdb148cf944fff79f699\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9ccee41603155e1ae29ca89df28ec1c12dd93e7d46098d032ecec1e8d8215b6e\\\"],\\\"sizeBytes\\\":272736506},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1d17bd2248667a524fcaafa91cdb351cad919be880335d6eb04ee24c658928c2\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:90d2319a9a2c6beea025f4be564f5ce39082fd4d7e74dada5b3012e790942555\\\"],\\\"sizeBytes\\\":255512859},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d8ec578bbd6716e84b3f12ae63e637d79aec1b16fb03556784b8ed5fce16d1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6cb6bdafd8554a7b8765137e45621fe33091c54b98ab962ff24e5f8c3fd2c531\\\"],\\\"sizeBytes\\\":254830869},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35d1ee61dada1d62dcc613c552ad9918bd7d2a6132a081d53de4775d3d16841c\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6beed8f25b71249945ab6789b94880b9c061dc53d3bb9f4cad17a5ca9de88e29\\\"],\\\"sizeBytes\\\":253435945},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:015b604a274fa1896122e0e4690585b3a83549f7c8670931a798446eec8963d0\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c487357748145bddf8d3d1f7036542da604a17f08f73602612838579d1bd3ac\\\"],\\\"sizeBytes\\\":252846435},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076da7d59107bba89a1430a080176af4f83052296230df56d51b724cf03a169e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e3a2f3c9f8d7a459ccd0719643fba2d050d158d387ade828c3d58813bc3b6e4\\\"],\\\"sizeBytes\\\":240112805},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:909d94acdf16bd26857849698608cb17db0c62ca3eab7d670ee858a6e839390e\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ead92f00c83a562232b8feac34089875e63ff76080039dc4c09e8ed9cc22d5fb\\\"],\\\"sizeBytes\\\":239854772},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b7540027380dafacdaf56cac1129e87b9081a45e3906f9363eb19d5a57ecf12\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e53129eef13d9deaac86d2e0f99a320d5b7b28d8347c5418169db98a679913de\\\"],\\\"sizeBytes\\\":229649173},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dba6a8cc4601d0c588a8dde112534f8e60a458bec75034ab9ed7dac6d45d0875\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e35f17762fc3f961f361749a5e9d544b17011c64187f85e085c763204f33a113\\\"],\\\"sizeBytes\\\":227367336},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2ed6cd765c62307186d486139e35584ce657840609d2147d464cae3481404155\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee7e710dffa2957f4ef74529a60af22f22f5e7846bcac2f8e8743cb57628b322\\\"],\\\"sizeBytes\\\":222920470},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7fb1140d1be7f58dafc56142241b6733f8b60b960b64dcb57585ef17d9cc459b\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f675d5827e7bef4552ea615e94b398c323676551997d51bec2e58778e962657\\\"],\\\"sizeBytes\\\":220121222},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1\\\",\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bfe8897012a86426822069e3423480528ca16f1610a60eefd6bc70e0e140d941\\\"],\\\"sizeBytes\\\":210996836}]}}\" for node \"ip-10-0-143-85.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ac4284c0-d934-4998-a228-c25a811a43dd.svc:9743/node?timeout=10s\": dial tcp 172.30.164.151:9743: connect: connection refused" Sep 07 17:10:56.000487 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:10:56.000483 2563 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count" Sep 07 17:10:57.338954 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:57.338920 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:10:57.418029 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:10:57.417996 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:00.375217 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:00.375179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:00.442303 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:00.442273 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:02.484155 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:02.484125 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cfdfbdc99-nltls"] Sep 07 17:11:02.496019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:02.495988 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77f5955fcd-j2ttz"] Sep 07 17:11:02.497429 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:02.497405 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cfdfbdc99-nltls"] Sep 07 17:11:02.560449 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:02.560406 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:02.659306 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:02.659271 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:02.763279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:02.763247 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:02.828959 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:02.828930 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:04.426463 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:04.426389 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5945742-663c-47cb-a119-ef0423cd37f0" path="/var/lib/kubelet/pods/f5945742-663c-47cb-a119-ef0423cd37f0/volumes" Sep 07 17:11:04.845951 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:04.845912 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:04.902669 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:04.902637 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:11:04.906292 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:04.906271 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-64868b84df-pqxn4" Sep 07 17:11:05.573901 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:05.573856 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:08.900684 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:08.900637 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:08.971474 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:08.971440 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:13.576603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:13.576566 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:13.717847 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:13.717801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:15.451614 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:15.451574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:15.552418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:15.552377 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:27.516826 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.516780 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77f5955fcd-j2ttz" podUID="bbb03ed7-076e-43de-b8a6-3e75db8bed15" containerName="console" containerID="cri-o://d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b" gracePeriod=15 Sep 07 17:11:27.760927 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.760902 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:11:27.922349 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922262 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-oauth-serving-cert\") pod \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " Sep 07 17:11:27.922349 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922304 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-trusted-ca-bundle\") pod \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " Sep 07 17:11:27.922349 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922347 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-config\") pod \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " Sep 07 17:11:27.922606 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922379 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-oauth-config\") pod \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " Sep 07 17:11:27.922606 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922445 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-service-ca\") pod \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " Sep 07 17:11:27.922606 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922493 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-serving-cert\") pod \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " Sep 07 17:11:27.922606 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922540 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xlj5\" (UniqueName: \"kubernetes.io/projected/bbb03ed7-076e-43de-b8a6-3e75db8bed15-kube-api-access-4xlj5\") pod \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\" (UID: \"bbb03ed7-076e-43de-b8a6-3e75db8bed15\") " Sep 07 17:11:27.922832 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922714 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bbb03ed7-076e-43de-b8a6-3e75db8bed15" (UID: "bbb03ed7-076e-43de-b8a6-3e75db8bed15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:11:27.922832 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922796 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-config" (OuterVolumeSpecName: "console-config") pod "bbb03ed7-076e-43de-b8a6-3e75db8bed15" (UID: "bbb03ed7-076e-43de-b8a6-3e75db8bed15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:11:27.922832 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922808 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bbb03ed7-076e-43de-b8a6-3e75db8bed15" (UID: "bbb03ed7-076e-43de-b8a6-3e75db8bed15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:11:27.922957 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.922867 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-service-ca" (OuterVolumeSpecName: "service-ca") pod "bbb03ed7-076e-43de-b8a6-3e75db8bed15" (UID: "bbb03ed7-076e-43de-b8a6-3e75db8bed15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:11:27.924771 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.924720 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bbb03ed7-076e-43de-b8a6-3e75db8bed15" (UID: "bbb03ed7-076e-43de-b8a6-3e75db8bed15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:27.924896 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.924840 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbb03ed7-076e-43de-b8a6-3e75db8bed15-kube-api-access-4xlj5" (OuterVolumeSpecName: "kube-api-access-4xlj5") pod "bbb03ed7-076e-43de-b8a6-3e75db8bed15" (UID: "bbb03ed7-076e-43de-b8a6-3e75db8bed15"). InnerVolumeSpecName "kube-api-access-4xlj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:11:27.924896 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:27.924848 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bbb03ed7-076e-43de-b8a6-3e75db8bed15" (UID: "bbb03ed7-076e-43de-b8a6-3e75db8bed15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:28.023299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.023244 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:28.023299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.023291 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-oauth-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:28.023299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.023302 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-service-ca\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:28.023299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.023310 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb03ed7-076e-43de-b8a6-3e75db8bed15-console-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:28.023299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.023319 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xlj5\" (UniqueName: \"kubernetes.io/projected/bbb03ed7-076e-43de-b8a6-3e75db8bed15-kube-api-access-4xlj5\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:28.023593 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.023328 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-oauth-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:28.023593 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.023337 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb03ed7-076e-43de-b8a6-3e75db8bed15-trusted-ca-bundle\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:28.236526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.236432 2563 generic.go:356] "Generic (PLEG): container finished" podID="bbb03ed7-076e-43de-b8a6-3e75db8bed15" containerID="d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b" exitCode=0 Sep 07 17:11:28.236526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.236492 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f5955fcd-j2ttz" Sep 07 17:11:28.236526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.236519 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f5955fcd-j2ttz" event={"ID":"bbb03ed7-076e-43de-b8a6-3e75db8bed15","Type":"ContainerDied","Data":"d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b"} Sep 07 17:11:28.236811 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.236589 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f5955fcd-j2ttz" event={"ID":"bbb03ed7-076e-43de-b8a6-3e75db8bed15","Type":"ContainerDied","Data":"aec1aed5fe32f3d6dbf07d6d360bc0e40e861a030c488ec9bf401c8b064d63e4"} Sep 07 17:11:28.236811 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.236610 2563 scope.go:117] "RemoveContainer" containerID="d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b" Sep 07 17:11:28.245140 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.245112 2563 scope.go:117] "RemoveContainer" containerID="d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b" Sep 07 17:11:28.245410 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:28.245393 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b\": container with ID starting with d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b not found: ID does not exist" containerID="d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b" Sep 07 17:11:28.245471 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.245418 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b"} err="failed to get container status \"d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b\": rpc error: code = NotFound desc = could not find container \"d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b\": container with ID starting with d92c2dbc11f9e6b7f7af35ea50b5b3f49cbebdaacb3d425fd244ed29e64d1f0b not found: ID does not exist" Sep 07 17:11:28.264351 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.264324 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77f5955fcd-j2ttz"] Sep 07 17:11:28.267426 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.267406 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77f5955fcd-j2ttz"] Sep 07 17:11:28.425783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:28.425751 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbb03ed7-076e-43de-b8a6-3e75db8bed15" path="/var/lib/kubelet/pods/bbb03ed7-076e-43de-b8a6-3e75db8bed15/volumes" Sep 07 17:11:39.587209 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:39.587175 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:40.658431 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.658392 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:11:40.659561 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.659502 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="alertmanager" containerID="cri-o://440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05" gracePeriod=120 Sep 07 17:11:40.660714 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.660197 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="prom-label-proxy" containerID="cri-o://dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e" gracePeriod=120 Sep 07 17:11:40.660714 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.660351 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-metric" containerID="cri-o://1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df" gracePeriod=120 Sep 07 17:11:40.660714 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.660447 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy" containerID="cri-o://022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0" gracePeriod=120 Sep 07 17:11:40.660714 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.660536 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-web" containerID="cri-o://4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db" gracePeriod=120 Sep 07 17:11:40.660714 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.660617 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="config-reloader" containerID="cri-o://215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e" gracePeriod=120 Sep 07 17:11:40.668126 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:40.668100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:41.288262 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288226 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerID="dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e" exitCode=0 Sep 07 17:11:41.288262 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288249 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerID="022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0" exitCode=0 Sep 07 17:11:41.288262 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288255 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerID="215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e" exitCode=0 Sep 07 17:11:41.288262 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288261 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerID="440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05" exitCode=0 Sep 07 17:11:41.288526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288280 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e"} Sep 07 17:11:41.288526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0"} Sep 07 17:11:41.288526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288357 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e"} Sep 07 17:11:41.288526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.288380 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05"} Sep 07 17:11:41.905087 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:41.905061 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.035797 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035763 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-main-db\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.035967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035807 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-cluster-tls-config\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.035967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035826 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-web-config\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.035967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035857 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-metrics-client-ca\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.035967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035893 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.035967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035918 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-web\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.035967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035940 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-volume\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.035975 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-tls-assets\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036000 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdhlh\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-kube-api-access-bdhlh\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036040 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-trusted-ca-bundle\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036085 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036116 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036141 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-out\") pod \"aa5e7d66-6765-4f04-954f-c7cddbead03e\" (UID: \"aa5e7d66-6765-4f04-954f-c7cddbead03e\") " Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036180 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:11:42.036340 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036294 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:11:42.036769 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036403 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-main-db\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.036769 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.036424 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-metrics-client-ca\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.037369 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.037317 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:11:42.039669 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.039544 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:42.039669 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.039593 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-kube-api-access-bdhlh" (OuterVolumeSpecName: "kube-api-access-bdhlh") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "kube-api-access-bdhlh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:11:42.039669 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.039632 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:42.039921 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.039861 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-out" (OuterVolumeSpecName: "config-out") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:11:42.039921 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.039870 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:42.040337 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.040316 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:11:42.040430 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.040377 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-volume" (OuterVolumeSpecName: "config-volume") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:42.040535 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.040515 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:42.042895 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.042868 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:42.049156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.049135 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-web-config" (OuterVolumeSpecName: "web-config") pod "aa5e7d66-6765-4f04-954f-c7cddbead03e" (UID: "aa5e7d66-6765-4f04-954f-c7cddbead03e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:11:42.137129 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137084 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137129 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137120 2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-volume\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137129 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137133 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-tls-assets\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137145 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdhlh\" (UniqueName: \"kubernetes.io/projected/aa5e7d66-6765-4f04-954f-c7cddbead03e-kube-api-access-bdhlh\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137158 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa5e7d66-6765-4f04-954f-c7cddbead03e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137170 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137182 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137195 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa5e7d66-6765-4f04-954f-c7cddbead03e-config-out\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137206 2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-cluster-tls-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137245 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-web-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.137383 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.137257 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa5e7d66-6765-4f04-954f-c7cddbead03e-secret-alertmanager-main-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:11:42.294953 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.294868 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerID="1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df" exitCode=0 Sep 07 17:11:42.294953 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.294894 2563 generic.go:356] "Generic (PLEG): container finished" podID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerID="4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db" exitCode=0 Sep 07 17:11:42.295123 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.294959 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.295123 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.294953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df"} Sep 07 17:11:42.295123 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.295102 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db"} Sep 07 17:11:42.295210 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.295149 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa5e7d66-6765-4f04-954f-c7cddbead03e","Type":"ContainerDied","Data":"490a2e72ae77b2f0ad476b39d32c4a223abf97d0bbb32dfb728b76b68c0fc2cd"} Sep 07 17:11:42.295210 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.295165 2563 scope.go:117] "RemoveContainer" containerID="dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e" Sep 07 17:11:42.303194 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.303172 2563 scope.go:117] "RemoveContainer" containerID="1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df" Sep 07 17:11:42.310396 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.310379 2563 scope.go:117] "RemoveContainer" containerID="022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0" Sep 07 17:11:42.316891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.316875 2563 scope.go:117] "RemoveContainer" containerID="4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db" Sep 07 17:11:42.323175 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.323155 2563 scope.go:117] "RemoveContainer" containerID="215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e" Sep 07 17:11:42.330056 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.330027 2563 scope.go:117] "RemoveContainer" containerID="440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05" Sep 07 17:11:42.332503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.332481 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:11:42.336371 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.336348 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:11:42.337839 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.337816 2563 scope.go:117] "RemoveContainer" containerID="a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db" Sep 07 17:11:42.344385 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.344368 2563 scope.go:117] "RemoveContainer" containerID="dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e" Sep 07 17:11:42.344648 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:42.344630 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e\": container with ID starting with dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e not found: ID does not exist" containerID="dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e" Sep 07 17:11:42.344719 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.344661 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e"} err="failed to get container status \"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e\": rpc error: code = NotFound desc = could not find container \"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e\": container with ID starting with dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e not found: ID does not exist" Sep 07 17:11:42.344719 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.344688 2563 scope.go:117] "RemoveContainer" containerID="1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df" Sep 07 17:11:42.345059 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:42.345043 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df\": container with ID starting with 1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df not found: ID does not exist" containerID="1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df" Sep 07 17:11:42.345109 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345064 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df"} err="failed to get container status \"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df\": rpc error: code = NotFound desc = could not find container \"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df\": container with ID starting with 1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df not found: ID does not exist" Sep 07 17:11:42.345109 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345077 2563 scope.go:117] "RemoveContainer" containerID="022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0" Sep 07 17:11:42.345334 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:42.345316 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0\": container with ID starting with 022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0 not found: ID does not exist" containerID="022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0" Sep 07 17:11:42.345377 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345339 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0"} err="failed to get container status \"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0\": rpc error: code = NotFound desc = could not find container \"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0\": container with ID starting with 022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0 not found: ID does not exist" Sep 07 17:11:42.345377 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345356 2563 scope.go:117] "RemoveContainer" containerID="4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db" Sep 07 17:11:42.345568 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:42.345549 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db\": container with ID starting with 4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db not found: ID does not exist" containerID="4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db" Sep 07 17:11:42.345607 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345572 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db"} err="failed to get container status \"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db\": rpc error: code = NotFound desc = could not find container \"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db\": container with ID starting with 4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db not found: ID does not exist" Sep 07 17:11:42.345607 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345585 2563 scope.go:117] "RemoveContainer" containerID="215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e" Sep 07 17:11:42.345804 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:42.345786 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e\": container with ID starting with 215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e not found: ID does not exist" containerID="215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e" Sep 07 17:11:42.345852 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345809 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e"} err="failed to get container status \"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e\": rpc error: code = NotFound desc = could not find container \"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e\": container with ID starting with 215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e not found: ID does not exist" Sep 07 17:11:42.345852 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.345823 2563 scope.go:117] "RemoveContainer" containerID="440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05" Sep 07 17:11:42.346068 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:42.346053 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05\": container with ID starting with 440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05 not found: ID does not exist" containerID="440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05" Sep 07 17:11:42.346104 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346072 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05"} err="failed to get container status \"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05\": rpc error: code = NotFound desc = could not find container \"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05\": container with ID starting with 440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05 not found: ID does not exist" Sep 07 17:11:42.346104 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346084 2563 scope.go:117] "RemoveContainer" containerID="a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db" Sep 07 17:11:42.346324 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:11:42.346307 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db\": container with ID starting with a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db not found: ID does not exist" containerID="a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db" Sep 07 17:11:42.346388 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346331 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db"} err="failed to get container status \"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db\": rpc error: code = NotFound desc = could not find container \"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db\": container with ID starting with a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db not found: ID does not exist" Sep 07 17:11:42.346388 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346350 2563 scope.go:117] "RemoveContainer" containerID="dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e" Sep 07 17:11:42.346575 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346557 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e"} err="failed to get container status \"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e\": rpc error: code = NotFound desc = could not find container \"dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e\": container with ID starting with dcedf068ac53186827ab9712d9c770bf1d038eaf1eedefd37a83dd874cc68c9e not found: ID does not exist" Sep 07 17:11:42.346618 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346577 2563 scope.go:117] "RemoveContainer" containerID="1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df" Sep 07 17:11:42.346845 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346826 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df"} err="failed to get container status \"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df\": rpc error: code = NotFound desc = could not find container \"1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df\": container with ID starting with 1ccddf70055c6dd8522861c105002f37a6c6024aaeac3301de05e3754807a4df not found: ID does not exist" Sep 07 17:11:42.346910 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.346846 2563 scope.go:117] "RemoveContainer" containerID="022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0" Sep 07 17:11:42.347051 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347037 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0"} err="failed to get container status \"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0\": rpc error: code = NotFound desc = could not find container \"022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0\": container with ID starting with 022b7416640a03b9efa80dfd70a05341efa276d8be11c31660dec22421c781c0 not found: ID does not exist" Sep 07 17:11:42.347097 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347051 2563 scope.go:117] "RemoveContainer" containerID="4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db" Sep 07 17:11:42.347288 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347267 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db"} err="failed to get container status \"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db\": rpc error: code = NotFound desc = could not find container \"4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db\": container with ID starting with 4bfbcaa3af81db4cbb854abf19ee0846893e95a5b9dbaf00526bd8cb9e74b7db not found: ID does not exist" Sep 07 17:11:42.347342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347289 2563 scope.go:117] "RemoveContainer" containerID="215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e" Sep 07 17:11:42.347492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347476 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e"} err="failed to get container status \"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e\": rpc error: code = NotFound desc = could not find container \"215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e\": container with ID starting with 215c87e4e67e136869885ef300cab1267dc59bc26b85183aa3618eb7a344150e not found: ID does not exist" Sep 07 17:11:42.347538 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347494 2563 scope.go:117] "RemoveContainer" containerID="440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05" Sep 07 17:11:42.347711 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347696 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05"} err="failed to get container status \"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05\": rpc error: code = NotFound desc = could not find container \"440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05\": container with ID starting with 440ff5a74c0fa2c4068564dfd8da9bc3cbd774b6d763da40a9acf09525589c05 not found: ID does not exist" Sep 07 17:11:42.347711 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347711 2563 scope.go:117] "RemoveContainer" containerID="a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db" Sep 07 17:11:42.347932 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.347916 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db"} err="failed to get container status \"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db\": rpc error: code = NotFound desc = could not find container \"a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db\": container with ID starting with a5aa7d69ac9d9bb3e4b7291fd2d848ad43355bf0e83630e2e06d0d47190340db not found: ID does not exist" Sep 07 17:11:42.412290 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412258 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:11:42.412521 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412507 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-web" Sep 07 17:11:42.412566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412526 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-web" Sep 07 17:11:42.412566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412535 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-metric" Sep 07 17:11:42.412566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412542 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-metric" Sep 07 17:11:42.412566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412554 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbb03ed7-076e-43de-b8a6-3e75db8bed15" containerName="console" Sep 07 17:11:42.412566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412560 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbb03ed7-076e-43de-b8a6-3e75db8bed15" containerName="console" Sep 07 17:11:42.412566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412567 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5945742-663c-47cb-a119-ef0423cd37f0" containerName="registry" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412573 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5945742-663c-47cb-a119-ef0423cd37f0" containerName="registry" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412580 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="prom-label-proxy" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412585 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="prom-label-proxy" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412594 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412602 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412614 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="init-config-reloader" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412619 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="init-config-reloader" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412625 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="config-reloader" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412630 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="config-reloader" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412638 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="alertmanager" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412643 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="alertmanager" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412686 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="alertmanager" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412694 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5945742-663c-47cb-a119-ef0423cd37f0" containerName="registry" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412701 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbb03ed7-076e-43de-b8a6-3e75db8bed15" containerName="console" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412707 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="config-reloader" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412712 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-web" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412717 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy-metric" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412724 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="kube-rbac-proxy" Sep 07 17:11:42.412791 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.412743 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" containerName="prom-label-proxy" Sep 07 17:11:42.417713 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.417696 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.424871 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.424845 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa5e7d66-6765-4f04-954f-c7cddbead03e" path="/var/lib/kubelet/pods/aa5e7d66-6765-4f04-954f-c7cddbead03e/volumes" Sep 07 17:11:42.435951 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.435927 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 07 17:11:42.436183 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.436167 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 07 17:11:42.436291 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.436273 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 07 17:11:42.436381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.436367 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 07 17:11:42.436748 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.436714 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 07 17:11:42.436845 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.436778 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 07 17:11:42.467394 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.467370 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 07 17:11:42.479852 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.479828 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 07 17:11:42.479852 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.479829 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-m5kbw\"" Sep 07 17:11:42.540063 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540030 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-config-volume\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540082 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540111 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540128 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-config-out\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540168 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540244 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540183 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540491 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540261 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540491 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540491 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540362 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540491 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540473 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsvp9\" (UniqueName: \"kubernetes.io/projected/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-kube-api-access-vsvp9\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540693 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.540693 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.540614 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-web-config\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.555581 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.555515 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 07 17:11:42.613768 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.613723 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:11:42.641820 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.641783 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.641834 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.641886 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvp9\" (UniqueName: \"kubernetes.io/projected/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-kube-api-access-vsvp9\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.641932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.641964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-web-config\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642190 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-config-volume\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642190 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642062 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642190 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642100 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642190 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642190 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642170 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-config-out\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.642418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.642283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.646639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.643767 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.646639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.644387 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.646639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.646454 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.647466 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.647435 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-config-volume\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.647936 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.647910 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.648020 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.647946 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.648794 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.648771 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.649763 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.649066 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.650325 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.650302 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.650966 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.650944 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.651051 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.651036 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-config-out\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.651279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.651259 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-web-config\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.671260 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.671234 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsvp9\" (UniqueName: \"kubernetes.io/projected/32c5aaaf-93f2-4b41-97da-b4f00a6df3b7-kube-api-access-vsvp9\") pod \"alertmanager-main-0\" (UID: \"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:42.726827 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:42.726788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 17:11:43.013517 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:43.013483 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 17:11:43.017280 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:11:43.017236 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c5aaaf_93f2_4b41_97da_b4f00a6df3b7.slice/crio-3fd0da2340de1dff2c7f6a5785e81037ee90cacfc918fc39140942d8aac3babb WatchSource:0}: Error finding container 3fd0da2340de1dff2c7f6a5785e81037ee90cacfc918fc39140942d8aac3babb: Status 404 returned error can't find the container with id 3fd0da2340de1dff2c7f6a5785e81037ee90cacfc918fc39140942d8aac3babb Sep 07 17:11:43.300833 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:43.300714 2563 generic.go:356] "Generic (PLEG): container finished" podID="32c5aaaf-93f2-4b41-97da-b4f00a6df3b7" containerID="637e8977aefb70a898df15bfef8b5b671c5711dd7b5474dc610fcf99b8af0528" exitCode=0 Sep 07 17:11:43.300833 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:43.300803 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerDied","Data":"637e8977aefb70a898df15bfef8b5b671c5711dd7b5474dc610fcf99b8af0528"} Sep 07 17:11:43.301007 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:43.300873 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerStarted","Data":"3fd0da2340de1dff2c7f6a5785e81037ee90cacfc918fc39140942d8aac3babb"} Sep 07 17:11:44.308276 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.308240 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerStarted","Data":"108ea4380d27b18ab80f000fd97daf3fbefa3f142fde6a9e8f0fae39201024e4"} Sep 07 17:11:44.308632 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.308308 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerStarted","Data":"cf0420c50d46eef46ae1353e7bc58eb558ed8f5328460290a7df8d856003bf66"} Sep 07 17:11:44.308632 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.308338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerStarted","Data":"10cd89e616cba3e45637f7ad4d8e8903e614d1ca1dbe117a806431aff4c91a76"} Sep 07 17:11:44.308632 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.308363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerStarted","Data":"8f06c1b4e0f51ae58666f6a6854e34ca0a840dafb34bcaca1ffb24851b024c99"} Sep 07 17:11:44.308632 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.308384 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerStarted","Data":"78f2da06a4bb2628299845f1e70cae6f05fe791b660b23b05cc50bfbf6bc1a2a"} Sep 07 17:11:44.308632 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.308407 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"32c5aaaf-93f2-4b41-97da-b4f00a6df3b7","Type":"ContainerStarted","Data":"1829b8fb2a8c6c85af3b7cf06db98a3e6d954426746aeb57943fb0aee8270d4b"} Sep 07 17:11:44.336862 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.336803 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.323960889 podStartE2EDuration="2.336786443s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.825554ms" isStatefulPod=false podCreationTimestamp="2026-09-07 17:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:11:44.335637968 +0000 UTC m=+212.408539314" watchObservedRunningTime="2026-09-07 17:11:44.336786443 +0000 UTC m=+212.409687776" Sep 07 17:11:44.717704 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.717620 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5d754ffdc-wr4dx"] Sep 07 17:11:44.720988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.720968 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.723174 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.723128 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-h4b9n\"" Sep 07 17:11:44.723469 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.723449 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 07 17:11:44.723778 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.723760 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 07 17:11:44.723998 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.723982 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 07 17:11:44.724174 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.724159 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 07 17:11:44.724616 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.724597 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 07 17:11:44.729105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.729080 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 07 17:11:44.736812 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.736777 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d754ffdc-wr4dx"] Sep 07 17:11:44.761128 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761087 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-federate-client-tls\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.761128 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761126 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-metrics-client-ca\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.761398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-serving-certs-ca-bundle\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.761398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761165 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.761398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761215 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.761398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761257 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-telemeter-client-tls\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.761398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-secret-telemeter-client\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.761398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.761367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pl86\" (UniqueName: \"kubernetes.io/projected/a7635e9f-0653-45a3-9929-436bc8593e15-kube-api-access-4pl86\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.861787 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.861740 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-federate-client-tls\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.861979 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.861912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-metrics-client-ca\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.861979 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.861967 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-serving-certs-ca-bundle\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.862098 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.861994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.862098 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.862027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.862098 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.862064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-telemeter-client-tls\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.862248 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.862102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-secret-telemeter-client\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.862248 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.862164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pl86\" (UniqueName: \"kubernetes.io/projected/a7635e9f-0653-45a3-9929-436bc8593e15-kube-api-access-4pl86\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.862715 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.862684 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-metrics-client-ca\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.863242 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.863211 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.863856 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.863828 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7635e9f-0653-45a3-9929-436bc8593e15-serving-certs-ca-bundle\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.864618 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.864590 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-telemeter-client-tls\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.864783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.864757 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-secret-telemeter-client\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.864889 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.864865 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.864943 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.864907 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7635e9f-0653-45a3-9929-436bc8593e15-federate-client-tls\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:44.871767 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:44.871740 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pl86\" (UniqueName: \"kubernetes.io/projected/a7635e9f-0653-45a3-9929-436bc8593e15-kube-api-access-4pl86\") pod \"telemeter-client-5d754ffdc-wr4dx\" (UID: \"a7635e9f-0653-45a3-9929-436bc8593e15\") " pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:45.033070 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:45.033024 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" Sep 07 17:11:45.336959 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:11:45.336918 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7635e9f_0653_45a3_9929_436bc8593e15.slice/crio-8be57cb248bb01fa309de7a06f08c71f4ae3f3600864d24bd15a501cb9d72d24 WatchSource:0}: Error finding container 8be57cb248bb01fa309de7a06f08c71f4ae3f3600864d24bd15a501cb9d72d24: Status 404 returned error can't find the container with id 8be57cb248bb01fa309de7a06f08c71f4ae3f3600864d24bd15a501cb9d72d24 Sep 07 17:11:45.341147 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:45.341119 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d754ffdc-wr4dx"] Sep 07 17:11:46.320061 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:46.320015 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" event={"ID":"a7635e9f-0653-45a3-9929-436bc8593e15","Type":"ContainerStarted","Data":"8be57cb248bb01fa309de7a06f08c71f4ae3f3600864d24bd15a501cb9d72d24"} Sep 07 17:11:47.326751 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.326637 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" event={"ID":"a7635e9f-0653-45a3-9929-436bc8593e15","Type":"ContainerStarted","Data":"c3c405cb84f80c041db58763a43037da5ab636af25896b40b5ca30e31e7c57c2"} Sep 07 17:11:47.326751 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.326708 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" event={"ID":"a7635e9f-0653-45a3-9929-436bc8593e15","Type":"ContainerStarted","Data":"22d56ebbf2093723c0c18ea77a9ca372012551bd11ca5d9f83054ce81b03c906"} Sep 07 17:11:47.326751 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.326750 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" event={"ID":"a7635e9f-0653-45a3-9929-436bc8593e15","Type":"ContainerStarted","Data":"cb21bf43ec5c6b1ed21472756b0ca1e05801c33e0b662566866ee646f59c6455"} Sep 07 17:11:47.350973 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.350918 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5d754ffdc-wr4dx" podStartSLOduration=1.670131094 podStartE2EDuration="3.350899221s" totalImagesPullingTime="1.680768127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:11:47.348989553 +0000 UTC m=+215.421890887" watchObservedRunningTime="2026-09-07 17:11:47.350899221 +0000 UTC m=+215.423800551" Sep 07 17:11:47.456113 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.456077 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cc598d6b9-jj7bd"] Sep 07 17:11:47.459299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.459278 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.473198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.473168 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc598d6b9-jj7bd"] Sep 07 17:11:47.480920 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.480888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-oauth-config\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.481083 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.480935 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-trusted-ca-bundle\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.481083 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.481013 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-config\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.481171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.481116 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7l72\" (UniqueName: \"kubernetes.io/projected/2241e2e3-d3ce-43dd-8d86-493743e003d4-kube-api-access-k7l72\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.481171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.481150 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-serving-cert\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.481264 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.481176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-oauth-serving-cert\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.481264 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.481227 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-service-ca\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.523820 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.523787 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:47.581689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.581601 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-config\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.581689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.581661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7l72\" (UniqueName: \"kubernetes.io/projected/2241e2e3-d3ce-43dd-8d86-493743e003d4-kube-api-access-k7l72\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.581689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.581684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-serving-cert\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.581704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-oauth-serving-cert\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.581724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-service-ca\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.581764 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-oauth-config\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.581787 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-trusted-ca-bundle\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582458 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.582432 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-service-ca\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582570 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.582439 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-config\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582570 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.582532 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-oauth-serving-cert\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.582695 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.582679 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-trusted-ca-bundle\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.584869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.584837 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-oauth-config\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.584961 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.584918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-serving-cert\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.586815 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.586794 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:11:47.589930 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.589907 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7l72\" (UniqueName: \"kubernetes.io/projected/2241e2e3-d3ce-43dd-8d86-493743e003d4-kube-api-access-k7l72\") pod \"console-cc598d6b9-jj7bd\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:47.768986 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:47.768944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:48.049648 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:48.049611 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc598d6b9-jj7bd"] Sep 07 17:11:48.052314 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:11:48.052286 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2241e2e3_d3ce_43dd_8d86_493743e003d4.slice/crio-f45ccf3893b22c60ae1c768e79a08d19fe8485c46679e92d698f8ff8f0c9e650 WatchSource:0}: Error finding container f45ccf3893b22c60ae1c768e79a08d19fe8485c46679e92d698f8ff8f0c9e650: Status 404 returned error can't find the container with id f45ccf3893b22c60ae1c768e79a08d19fe8485c46679e92d698f8ff8f0c9e650 Sep 07 17:11:48.331828 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:48.331701 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc598d6b9-jj7bd" event={"ID":"2241e2e3-d3ce-43dd-8d86-493743e003d4","Type":"ContainerStarted","Data":"584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a"} Sep 07 17:11:48.331828 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:48.331821 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc598d6b9-jj7bd" event={"ID":"2241e2e3-d3ce-43dd-8d86-493743e003d4","Type":"ContainerStarted","Data":"f45ccf3893b22c60ae1c768e79a08d19fe8485c46679e92d698f8ff8f0c9e650"} Sep 07 17:11:48.350890 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:48.350831 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cc598d6b9-jj7bd" podStartSLOduration=1.350805207 podStartE2EDuration="1.350805207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:11:48.350422321 +0000 UTC m=+216.423323656" watchObservedRunningTime="2026-09-07 17:11:48.350805207 +0000 UTC m=+216.423706541" Sep 07 17:11:57.769961 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:57.769917 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:57.770438 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:57.769991 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:57.774750 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:57.774706 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:58.373507 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:58.373477 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:11:58.649967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:58.649879 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6886fd7f-dp44v"] Sep 07 17:11:58.773974 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:11:58.773937 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:12:00.064326 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:00.064261 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:12:03.636984 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:03.636951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:12:03.695647 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:03.695608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:12:23.673901 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.673854 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d6886fd7f-dp44v" podUID="34314299-5ca7-46ec-9159-8568af291e23" containerName="console" containerID="cri-o://1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee" gracePeriod=15 Sep 07 17:12:23.914469 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.914442 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:12:23.987385 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987282 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-oauth-serving-cert\") pod \"34314299-5ca7-46ec-9159-8568af291e23\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " Sep 07 17:12:23.987385 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987329 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-oauth-config\") pod \"34314299-5ca7-46ec-9159-8568af291e23\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " Sep 07 17:12:23.987614 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987390 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-console-config\") pod \"34314299-5ca7-46ec-9159-8568af291e23\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " Sep 07 17:12:23.987614 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987427 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw9wr\" (UniqueName: \"kubernetes.io/projected/34314299-5ca7-46ec-9159-8568af291e23-kube-api-access-nw9wr\") pod \"34314299-5ca7-46ec-9159-8568af291e23\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " Sep 07 17:12:23.987614 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987461 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-service-ca\") pod \"34314299-5ca7-46ec-9159-8568af291e23\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " Sep 07 17:12:23.987614 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987489 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-trusted-ca-bundle\") pod \"34314299-5ca7-46ec-9159-8568af291e23\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " Sep 07 17:12:23.987614 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987536 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-serving-cert\") pod \"34314299-5ca7-46ec-9159-8568af291e23\" (UID: \"34314299-5ca7-46ec-9159-8568af291e23\") " Sep 07 17:12:23.987883 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987808 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "34314299-5ca7-46ec-9159-8568af291e23" (UID: "34314299-5ca7-46ec-9159-8568af291e23"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:12:23.987883 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987859 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-console-config" (OuterVolumeSpecName: "console-config") pod "34314299-5ca7-46ec-9159-8568af291e23" (UID: "34314299-5ca7-46ec-9159-8568af291e23"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:12:23.987993 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987871 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-service-ca" (OuterVolumeSpecName: "service-ca") pod "34314299-5ca7-46ec-9159-8568af291e23" (UID: "34314299-5ca7-46ec-9159-8568af291e23"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:12:23.987993 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.987930 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "34314299-5ca7-46ec-9159-8568af291e23" (UID: "34314299-5ca7-46ec-9159-8568af291e23"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:12:23.989475 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.989453 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "34314299-5ca7-46ec-9159-8568af291e23" (UID: "34314299-5ca7-46ec-9159-8568af291e23"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:12:23.989605 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.989585 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "34314299-5ca7-46ec-9159-8568af291e23" (UID: "34314299-5ca7-46ec-9159-8568af291e23"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:12:23.989605 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:23.989588 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34314299-5ca7-46ec-9159-8568af291e23-kube-api-access-nw9wr" (OuterVolumeSpecName: "kube-api-access-nw9wr") pod "34314299-5ca7-46ec-9159-8568af291e23" (UID: "34314299-5ca7-46ec-9159-8568af291e23"). InnerVolumeSpecName "kube-api-access-nw9wr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:12:24.088346 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.088304 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw9wr\" (UniqueName: \"kubernetes.io/projected/34314299-5ca7-46ec-9159-8568af291e23-kube-api-access-nw9wr\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:12:24.088346 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.088336 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-service-ca\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:12:24.088346 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.088349 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-trusted-ca-bundle\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:12:24.088584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.088360 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:12:24.088584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.088372 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-oauth-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:12:24.088584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.088383 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34314299-5ca7-46ec-9159-8568af291e23-console-oauth-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:12:24.088584 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.088394 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34314299-5ca7-46ec-9159-8568af291e23-console-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:12:24.465456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.465422 2563 generic.go:356] "Generic (PLEG): container finished" podID="34314299-5ca7-46ec-9159-8568af291e23" containerID="1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee" exitCode=0 Sep 07 17:12:24.465610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.465465 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6886fd7f-dp44v" event={"ID":"34314299-5ca7-46ec-9159-8568af291e23","Type":"ContainerDied","Data":"1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee"} Sep 07 17:12:24.465610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.465501 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6886fd7f-dp44v" event={"ID":"34314299-5ca7-46ec-9159-8568af291e23","Type":"ContainerDied","Data":"e26ba889f43e15e1f7e0828239c3cd1e321489524befc0e75c81322b7df11011"} Sep 07 17:12:24.465610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.465520 2563 scope.go:117] "RemoveContainer" containerID="1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee" Sep 07 17:12:24.465610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.465467 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6886fd7f-dp44v" Sep 07 17:12:24.473524 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.473498 2563 scope.go:117] "RemoveContainer" containerID="1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee" Sep 07 17:12:24.473808 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:12:24.473784 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\": container with ID starting with 1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee not found: ID does not exist" containerID="1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee" Sep 07 17:12:24.473910 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.473814 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee"} err="failed to get container status \"1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\": rpc error: code = NotFound desc = could not find container \"1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee\": container with ID starting with 1093a0724f4a639da55a8e3b522ad7867bd62a0e52a01b1a280e907780d7bcee not found: ID does not exist" Sep 07 17:12:24.480816 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.480788 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6886fd7f-dp44v"] Sep 07 17:12:24.484296 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:24.484270 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d6886fd7f-dp44v"] Sep 07 17:12:26.425187 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:26.425149 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34314299-5ca7-46ec-9159-8568af291e23" path="/var/lib/kubelet/pods/34314299-5ca7-46ec-9159-8568af291e23/volumes" Sep 07 17:12:54.868118 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:54.867999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:12:54.919751 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:12:54.919702 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:06.962943 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:06.962910 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:07.037323 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:07.037290 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:09.841141 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.841107 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5747c94b8b-7njdw"] Sep 07 17:13:09.841514 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.841388 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34314299-5ca7-46ec-9159-8568af291e23" containerName="console" Sep 07 17:13:09.841514 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.841399 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34314299-5ca7-46ec-9159-8568af291e23" containerName="console" Sep 07 17:13:09.841514 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.841454 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34314299-5ca7-46ec-9159-8568af291e23" containerName="console" Sep 07 17:13:09.844224 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.844208 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:09.854834 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.854809 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5747c94b8b-7njdw"] Sep 07 17:13:09.879319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.879270 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:09.939483 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.939453 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:09.952508 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.952485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-trusted-ca-bundle\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:09.952589 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.952527 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-console-config\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:09.952589 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.952549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkbwl\" (UniqueName: \"kubernetes.io/projected/1aba850e-54bb-4030-9af6-066450707767-kube-api-access-rkbwl\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:09.952654 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.952606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-oauth-config\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:09.952654 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.952627 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-oauth-serving-cert\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:09.952721 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.952674 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-service-ca\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:09.952721 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:09.952693 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-serving-cert\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.053894 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.053860 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-trusted-ca-bundle\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054062 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.053909 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-console-config\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054062 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.053933 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkbwl\" (UniqueName: \"kubernetes.io/projected/1aba850e-54bb-4030-9af6-066450707767-kube-api-access-rkbwl\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054062 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.053964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-oauth-config\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054062 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.053978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-oauth-serving-cert\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054062 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.054004 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-service-ca\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054321 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.054067 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-serving-cert\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054747 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.054698 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-console-config\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054868 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.054705 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-oauth-serving-cert\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054868 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.054776 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-service-ca\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.054868 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.054852 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-trusted-ca-bundle\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.056537 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.056511 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-serving-cert\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.056629 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.056538 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-oauth-config\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.062782 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.062706 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkbwl\" (UniqueName: \"kubernetes.io/projected/1aba850e-54bb-4030-9af6-066450707767-kube-api-access-rkbwl\") pod \"console-5747c94b8b-7njdw\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.152432 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.152346 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:10.429172 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.429101 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5747c94b8b-7njdw"] Sep 07 17:13:10.432186 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:13:10.432160 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aba850e_54bb_4030_9af6_066450707767.slice/crio-cd46c8d40e01ef438c4be9b66b1a94d8534c7bb782d902728183ef8934fd7319 WatchSource:0}: Error finding container cd46c8d40e01ef438c4be9b66b1a94d8534c7bb782d902728183ef8934fd7319: Status 404 returned error can't find the container with id cd46c8d40e01ef438c4be9b66b1a94d8534c7bb782d902728183ef8934fd7319 Sep 07 17:13:10.631184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.631150 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747c94b8b-7njdw" event={"ID":"1aba850e-54bb-4030-9af6-066450707767","Type":"ContainerStarted","Data":"0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5"} Sep 07 17:13:10.631347 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.631198 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747c94b8b-7njdw" event={"ID":"1aba850e-54bb-4030-9af6-066450707767","Type":"ContainerStarted","Data":"cd46c8d40e01ef438c4be9b66b1a94d8534c7bb782d902728183ef8934fd7319"} Sep 07 17:13:10.648056 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:10.647998 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5747c94b8b-7njdw" podStartSLOduration=1.647981812 podStartE2EDuration="1.647981812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:13:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:13:10.64793162 +0000 UTC m=+298.720832966" watchObservedRunningTime="2026-09-07 17:13:10.647981812 +0000 UTC m=+298.720883145" Sep 07 17:13:12.321522 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:12.321488 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:12.368626 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:12.368596 2563 kubelet.go:1648] "Image garbage collection succeeded" Sep 07 17:13:12.422599 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:12.422408 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:12.492875 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:12.492842 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:20.153363 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:20.153321 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:20.153786 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:20.153384 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:20.157972 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:20.157950 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:20.671373 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:20.671346 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:13:20.727222 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:20.727188 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc598d6b9-jj7bd"] Sep 07 17:13:20.851845 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:20.851815 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:20.952158 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:20.952067 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:21.013929 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:21.013895 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:13:45.747250 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:45.747202 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cc598d6b9-jj7bd" podUID="2241e2e3-d3ce-43dd-8d86-493743e003d4" containerName="console" containerID="cri-o://584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a" gracePeriod=15 Sep 07 17:13:45.981893 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:45.981863 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:13:46.051943 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.051864 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-serving-cert\") pod \"2241e2e3-d3ce-43dd-8d86-493743e003d4\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " Sep 07 17:13:46.051943 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.051917 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-oauth-config\") pod \"2241e2e3-d3ce-43dd-8d86-493743e003d4\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " Sep 07 17:13:46.051943 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.051941 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-config\") pod \"2241e2e3-d3ce-43dd-8d86-493743e003d4\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " Sep 07 17:13:46.052198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.051965 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-oauth-serving-cert\") pod \"2241e2e3-d3ce-43dd-8d86-493743e003d4\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " Sep 07 17:13:46.052198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.052005 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7l72\" (UniqueName: \"kubernetes.io/projected/2241e2e3-d3ce-43dd-8d86-493743e003d4-kube-api-access-k7l72\") pod \"2241e2e3-d3ce-43dd-8d86-493743e003d4\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " Sep 07 17:13:46.052198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.052174 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-trusted-ca-bundle\") pod \"2241e2e3-d3ce-43dd-8d86-493743e003d4\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " Sep 07 17:13:46.052344 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.052223 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-service-ca\") pod \"2241e2e3-d3ce-43dd-8d86-493743e003d4\" (UID: \"2241e2e3-d3ce-43dd-8d86-493743e003d4\") " Sep 07 17:13:46.052432 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.052404 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-config" (OuterVolumeSpecName: "console-config") pod "2241e2e3-d3ce-43dd-8d86-493743e003d4" (UID: "2241e2e3-d3ce-43dd-8d86-493743e003d4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:13:46.052487 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.052441 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2241e2e3-d3ce-43dd-8d86-493743e003d4" (UID: "2241e2e3-d3ce-43dd-8d86-493743e003d4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:13:46.052570 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.052541 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2241e2e3-d3ce-43dd-8d86-493743e003d4" (UID: "2241e2e3-d3ce-43dd-8d86-493743e003d4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:13:46.052792 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.052769 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-service-ca" (OuterVolumeSpecName: "service-ca") pod "2241e2e3-d3ce-43dd-8d86-493743e003d4" (UID: "2241e2e3-d3ce-43dd-8d86-493743e003d4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:13:46.054037 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.054015 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2241e2e3-d3ce-43dd-8d86-493743e003d4" (UID: "2241e2e3-d3ce-43dd-8d86-493743e003d4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:13:46.054471 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.054454 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2241e2e3-d3ce-43dd-8d86-493743e003d4" (UID: "2241e2e3-d3ce-43dd-8d86-493743e003d4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:13:46.054471 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.054458 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2241e2e3-d3ce-43dd-8d86-493743e003d4-kube-api-access-k7l72" (OuterVolumeSpecName: "kube-api-access-k7l72") pod "2241e2e3-d3ce-43dd-8d86-493743e003d4" (UID: "2241e2e3-d3ce-43dd-8d86-493743e003d4"). InnerVolumeSpecName "kube-api-access-k7l72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:13:46.153071 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.153022 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7l72\" (UniqueName: \"kubernetes.io/projected/2241e2e3-d3ce-43dd-8d86-493743e003d4-kube-api-access-k7l72\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:13:46.153071 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.153064 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-trusted-ca-bundle\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:13:46.153071 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.153074 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-service-ca\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:13:46.153071 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.153083 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:13:46.153071 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.153093 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-oauth-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:13:46.153375 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.153103 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-console-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:13:46.153375 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.153110 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2241e2e3-d3ce-43dd-8d86-493743e003d4-oauth-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:13:46.762013 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.761976 2563 generic.go:356] "Generic (PLEG): container finished" podID="2241e2e3-d3ce-43dd-8d86-493743e003d4" containerID="584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a" exitCode=0 Sep 07 17:13:46.762492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.762025 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc598d6b9-jj7bd" Sep 07 17:13:46.762492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.762068 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc598d6b9-jj7bd" event={"ID":"2241e2e3-d3ce-43dd-8d86-493743e003d4","Type":"ContainerDied","Data":"584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a"} Sep 07 17:13:46.762492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.762129 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc598d6b9-jj7bd" event={"ID":"2241e2e3-d3ce-43dd-8d86-493743e003d4","Type":"ContainerDied","Data":"f45ccf3893b22c60ae1c768e79a08d19fe8485c46679e92d698f8ff8f0c9e650"} Sep 07 17:13:46.762492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.762162 2563 scope.go:117] "RemoveContainer" containerID="584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a" Sep 07 17:13:46.769877 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.769861 2563 scope.go:117] "RemoveContainer" containerID="584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a" Sep 07 17:13:46.770159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:13:46.770141 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a\": container with ID starting with 584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a not found: ID does not exist" containerID="584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a" Sep 07 17:13:46.770209 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.770167 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a"} err="failed to get container status \"584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a\": rpc error: code = NotFound desc = could not find container \"584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a\": container with ID starting with 584b184aa46b432ffcd162ce008352af9b583cf71d2b555e621ea6fe16430e5a not found: ID does not exist" Sep 07 17:13:46.778806 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.778780 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc598d6b9-jj7bd"] Sep 07 17:13:46.782489 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:46.782466 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cc598d6b9-jj7bd"] Sep 07 17:13:48.425503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:13:48.425469 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2241e2e3-d3ce-43dd-8d86-493743e003d4" path="/var/lib/kubelet/pods/2241e2e3-d3ce-43dd-8d86-493743e003d4/volumes" Sep 07 17:14:54.498391 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:54.498356 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:14:54.568740 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:54.568696 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:14:55.342671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.342633 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:14:55.368419 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.368390 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj"] Sep 07 17:14:55.368674 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.368663 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2241e2e3-d3ce-43dd-8d86-493743e003d4" containerName="console" Sep 07 17:14:55.368715 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.368676 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2241e2e3-d3ce-43dd-8d86-493743e003d4" containerName="console" Sep 07 17:14:55.368778 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.368737 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2241e2e3-d3ce-43dd-8d86-493743e003d4" containerName="console" Sep 07 17:14:55.371706 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.371687 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:14:55.373505 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.373485 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 07 17:14:55.373807 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.373787 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 07 17:14:55.373865 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.373787 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-wnm4j\"" Sep 07 17:14:55.381987 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.381966 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj"] Sep 07 17:14:55.487839 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.487802 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr8ph\" (UniqueName: \"kubernetes.io/projected/0696698b-9f70-4540-aa3a-e85f49c513c3-kube-api-access-jr8ph\") pod \"feast-operator-controller-manager-695d6b979d-p8gkj\" (UID: \"0696698b-9f70-4540-aa3a-e85f49c513c3\") " pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:14:55.588861 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.588823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr8ph\" (UniqueName: \"kubernetes.io/projected/0696698b-9f70-4540-aa3a-e85f49c513c3-kube-api-access-jr8ph\") pod \"feast-operator-controller-manager-695d6b979d-p8gkj\" (UID: \"0696698b-9f70-4540-aa3a-e85f49c513c3\") " pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:14:55.597167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.597099 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr8ph\" (UniqueName: \"kubernetes.io/projected/0696698b-9f70-4540-aa3a-e85f49c513c3-kube-api-access-jr8ph\") pod \"feast-operator-controller-manager-695d6b979d-p8gkj\" (UID: \"0696698b-9f70-4540-aa3a-e85f49c513c3\") " pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:14:55.681833 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.681793 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:14:55.952314 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.952173 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj"] Sep 07 17:14:55.954958 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:14:55.954930 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0696698b_9f70_4540_aa3a_e85f49c513c3.slice/crio-5036a3ab8902b40a916b6aa95374406d9a03ad5191a4658673aaae3e8ab2a986 WatchSource:0}: Error finding container 5036a3ab8902b40a916b6aa95374406d9a03ad5191a4658673aaae3e8ab2a986: Status 404 returned error can't find the container with id 5036a3ab8902b40a916b6aa95374406d9a03ad5191a4658673aaae3e8ab2a986 Sep 07 17:14:55.956886 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:55.956870 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 17:14:56.013297 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:56.013260 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" event={"ID":"0696698b-9f70-4540-aa3a-e85f49c513c3","Type":"ContainerStarted","Data":"5036a3ab8902b40a916b6aa95374406d9a03ad5191a4658673aaae3e8ab2a986"} Sep 07 17:14:59.027486 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:59.027447 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" event={"ID":"0696698b-9f70-4540-aa3a-e85f49c513c3","Type":"ContainerStarted","Data":"e850e2692bf7ee9d1a25a53cf3dfc044ac28b7a22f3e04a9e50892ab4b52b58e"} Sep 07 17:14:59.027896 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:59.027629 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:14:59.045740 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:14:59.045678 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" podStartSLOduration=1.783470831 podStartE2EDuration="4.045663605s" totalImagesPullingTime="2.262192774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:14:59.044302579 +0000 UTC m=+407.117203912" watchObservedRunningTime="2026-09-07 17:14:59.045663605 +0000 UTC m=+407.118564938" Sep 07 17:15:06.518054 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:06.518020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:10.033143 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:10.033103 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:15:22.450044 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.450009 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b4ff89f6-ssg5g"] Sep 07 17:15:22.453199 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.453180 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.467741 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.464521 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b4ff89f6-ssg5g"] Sep 07 17:15:22.516742 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.516656 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:22.618049 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.618014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:22.621036 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.621013 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-service-ca\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.621106 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.621058 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-config\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.621184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.621163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q48ns\" (UniqueName: \"kubernetes.io/projected/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-kube-api-access-q48ns\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.621241 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.621218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-serving-cert\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.621290 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.621270 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-oauth-serving-cert\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.621354 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.621309 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-oauth-config\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.621354 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.621343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-trusted-ca-bundle\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.722177 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.722071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q48ns\" (UniqueName: \"kubernetes.io/projected/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-kube-api-access-q48ns\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.722177 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.722142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-serving-cert\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.722177 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.722184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-oauth-serving-cert\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.722446 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.722213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-oauth-config\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.722446 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.722245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-trusted-ca-bundle\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.722446 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.722279 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-service-ca\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.722446 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.722321 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-config\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.723130 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.723103 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-config\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.723255 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.723107 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-oauth-serving-cert\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.723255 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.723113 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-service-ca\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.723255 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.723210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-trusted-ca-bundle\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.724694 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.724668 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-serving-cert\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.724805 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.724691 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-console-oauth-config\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.731427 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.731406 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q48ns\" (UniqueName: \"kubernetes.io/projected/ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c-kube-api-access-q48ns\") pod \"console-77b4ff89f6-ssg5g\" (UID: \"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c\") " pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:22.761935 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:22.761906 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:23.032914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:23.032886 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b4ff89f6-ssg5g"] Sep 07 17:15:23.036392 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:15:23.036364 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2bcfbe_52f2_4a5d_b07f_4e5b77870d7c.slice/crio-5f97bcd4fdbfeafb148ddb28d8d0076db050577afcc87f6448d8b93653eb443b WatchSource:0}: Error finding container 5f97bcd4fdbfeafb148ddb28d8d0076db050577afcc87f6448d8b93653eb443b: Status 404 returned error can't find the container with id 5f97bcd4fdbfeafb148ddb28d8d0076db050577afcc87f6448d8b93653eb443b Sep 07 17:15:23.115772 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:23.115571 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b4ff89f6-ssg5g" event={"ID":"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c","Type":"ContainerStarted","Data":"9f556fd548269007a08123156d1540e630a6ea84bd926ce8b101491d72a099f7"} Sep 07 17:15:23.115772 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:23.115636 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b4ff89f6-ssg5g" event={"ID":"ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c","Type":"ContainerStarted","Data":"5f97bcd4fdbfeafb148ddb28d8d0076db050577afcc87f6448d8b93653eb443b"} Sep 07 17:15:23.133557 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:23.133508 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b4ff89f6-ssg5g" podStartSLOduration=1.133491302 podStartE2EDuration="1.133491302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:15:23.131495322 +0000 UTC m=+431.204396655" watchObservedRunningTime="2026-09-07 17:15:23.133491302 +0000 UTC m=+431.206392634" Sep 07 17:15:32.382150 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:32.382108 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:32.481648 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:32.481616 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:32.762912 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:32.762878 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:32.763104 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:32.762946 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:32.767558 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:32.767536 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:33.050608 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.050502 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:33.148405 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.148330 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:33.157987 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.157957 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b4ff89f6-ssg5g" Sep 07 17:15:33.209589 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.209558 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf"] Sep 07 17:15:33.217477 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.217447 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.219855 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.219829 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 07 17:15:33.220235 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.220215 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 17:15:33.220394 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.220364 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-kkj7z\"" Sep 07 17:15:33.220500 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.220217 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 07 17:15:33.220563 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.220216 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 17:15:33.220628 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.220263 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 07 17:15:33.220686 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.220647 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 07 17:15:33.220816 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.220793 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 07 17:15:33.226899 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.226866 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5747c94b8b-7njdw"] Sep 07 17:15:33.232806 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.232778 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf"] Sep 07 17:15:33.264362 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.264328 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:33.304049 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.303972 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.304049 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.304004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-ui-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.304224 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.304074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-online-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.304224 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.304089 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/114e8c2c-8237-416e-a727-815c4faccaf0-feast-data\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.304224 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.304108 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-offline-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.304224 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.304143 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7zqs\" (UniqueName: \"kubernetes.io/projected/114e8c2c-8237-416e-a727-815c4faccaf0-kube-api-access-j7zqs\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.344461 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.344427 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:33.405082 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.405043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.405603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.405091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-ui-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.405603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.405178 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-online-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.405603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.405205 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/114e8c2c-8237-416e-a727-815c4faccaf0-feast-data\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.405603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.405245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-offline-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.405603 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:15:33.405271 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 07 17:15:33.405603 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:15:33.405366 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls podName:114e8c2c-8237-416e-a727-815c4faccaf0 nodeName:}" failed. No retries permitted until 2026-09-07 17:15:33.905344401 +0000 UTC m=+441.978245718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls") pod "feast-simple-feast-setup-7cc9474b77-zfnvf" (UID: "114e8c2c-8237-416e-a727-815c4faccaf0") : secret "feast-simple-feast-setup-registry-tls" not found Sep 07 17:15:33.405603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.405275 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7zqs\" (UniqueName: \"kubernetes.io/projected/114e8c2c-8237-416e-a727-815c4faccaf0-kube-api-access-j7zqs\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.406017 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.405628 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/114e8c2c-8237-416e-a727-815c4faccaf0-feast-data\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.407824 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.407794 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-offline-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.407950 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.407852 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-ui-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.408070 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.408052 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-online-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.423188 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.418763 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7zqs\" (UniqueName: \"kubernetes.io/projected/114e8c2c-8237-416e-a727-815c4faccaf0-kube-api-access-j7zqs\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.442868 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.442831 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:33.544845 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.544806 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:33.643061 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.642966 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:15:33.910184 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.910088 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:33.912415 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:33.912387 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-zfnvf\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:34.136604 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:34.136569 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:15:34.418802 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:34.418760 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf"] Sep 07 17:15:34.421518 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:15:34.421494 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod114e8c2c_8237_416e_a727_815c4faccaf0.slice/crio-c47516488ec862cc9bf69a9a86941fed3e0138d402221996535c6ab0015a1b30 WatchSource:0}: Error finding container c47516488ec862cc9bf69a9a86941fed3e0138d402221996535c6ab0015a1b30: Status 404 returned error can't find the container with id c47516488ec862cc9bf69a9a86941fed3e0138d402221996535c6ab0015a1b30 Sep 07 17:15:35.163010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:35.162966 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerStarted","Data":"c47516488ec862cc9bf69a9a86941fed3e0138d402221996535c6ab0015a1b30"} Sep 07 17:15:58.250607 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:15:58.250560 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5747c94b8b-7njdw" podUID="1aba850e-54bb-4030-9af6-066450707767" containerName="console" containerID="cri-o://0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5" gracePeriod=15 Sep 07 17:16:00.669107 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:00.669071 2563 patch_prober.go:28] interesting pod/console-5747c94b8b-7njdw container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" start-of-body= Sep 07 17:16:00.669640 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:00.669149 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-5747c94b8b-7njdw" podUID="1aba850e-54bb-4030-9af6-066450707767" containerName="console" probeResult="failure" output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" Sep 07 17:16:02.867341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.867316 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:16:02.971189 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971111 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-console-config\") pod \"1aba850e-54bb-4030-9af6-066450707767\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " Sep 07 17:16:02.971189 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971152 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-serving-cert\") pod \"1aba850e-54bb-4030-9af6-066450707767\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " Sep 07 17:16:02.971189 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971181 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-trusted-ca-bundle\") pod \"1aba850e-54bb-4030-9af6-066450707767\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " Sep 07 17:16:02.971420 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971313 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-service-ca\") pod \"1aba850e-54bb-4030-9af6-066450707767\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " Sep 07 17:16:02.971420 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971384 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-oauth-config\") pod \"1aba850e-54bb-4030-9af6-066450707767\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " Sep 07 17:16:02.971527 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971424 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkbwl\" (UniqueName: \"kubernetes.io/projected/1aba850e-54bb-4030-9af6-066450707767-kube-api-access-rkbwl\") pod \"1aba850e-54bb-4030-9af6-066450707767\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " Sep 07 17:16:02.971527 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971451 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-oauth-serving-cert\") pod \"1aba850e-54bb-4030-9af6-066450707767\" (UID: \"1aba850e-54bb-4030-9af6-066450707767\") " Sep 07 17:16:02.971527 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971489 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-console-config" (OuterVolumeSpecName: "console-config") pod "1aba850e-54bb-4030-9af6-066450707767" (UID: "1aba850e-54bb-4030-9af6-066450707767"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:16:02.971687 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971582 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1aba850e-54bb-4030-9af6-066450707767" (UID: "1aba850e-54bb-4030-9af6-066450707767"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:16:02.971770 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971676 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-service-ca" (OuterVolumeSpecName: "service-ca") pod "1aba850e-54bb-4030-9af6-066450707767" (UID: "1aba850e-54bb-4030-9af6-066450707767"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:16:02.971770 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971715 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-console-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:16:02.971878 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971850 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-trusted-ca-bundle\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:16:02.971931 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.971910 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1aba850e-54bb-4030-9af6-066450707767" (UID: "1aba850e-54bb-4030-9af6-066450707767"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:16:02.973482 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.973460 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1aba850e-54bb-4030-9af6-066450707767" (UID: "1aba850e-54bb-4030-9af6-066450707767"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:16:02.973604 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.973587 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aba850e-54bb-4030-9af6-066450707767-kube-api-access-rkbwl" (OuterVolumeSpecName: "kube-api-access-rkbwl") pod "1aba850e-54bb-4030-9af6-066450707767" (UID: "1aba850e-54bb-4030-9af6-066450707767"). InnerVolumeSpecName "kube-api-access-rkbwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:16:02.973757 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:02.973738 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1aba850e-54bb-4030-9af6-066450707767" (UID: "1aba850e-54bb-4030-9af6-066450707767"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:16:03.073449 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.073353 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-oauth-config\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:16:03.073449 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.073385 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkbwl\" (UniqueName: \"kubernetes.io/projected/1aba850e-54bb-4030-9af6-066450707767-kube-api-access-rkbwl\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:16:03.073449 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.073399 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-oauth-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:16:03.073449 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.073413 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1aba850e-54bb-4030-9af6-066450707767-console-serving-cert\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:16:03.073449 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.073426 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba850e-54bb-4030-9af6-066450707767-service-ca\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:16:03.288179 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.288143 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerStarted","Data":"891d3e198510813aa45fcbb63fe7049abbef1ddd435e94e463a7da80d7d0e30d"} Sep 07 17:16:03.289548 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.289524 2563 generic.go:356] "Generic (PLEG): container finished" podID="1aba850e-54bb-4030-9af6-066450707767" containerID="0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5" exitCode=0 Sep 07 17:16:03.289661 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.289557 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747c94b8b-7njdw" event={"ID":"1aba850e-54bb-4030-9af6-066450707767","Type":"ContainerDied","Data":"0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5"} Sep 07 17:16:03.289661 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.289558 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747c94b8b-7njdw" Sep 07 17:16:03.289661 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.289601 2563 scope.go:117] "RemoveContainer" containerID="0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5" Sep 07 17:16:03.289819 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.289591 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747c94b8b-7njdw" event={"ID":"1aba850e-54bb-4030-9af6-066450707767","Type":"ContainerDied","Data":"cd46c8d40e01ef438c4be9b66b1a94d8534c7bb782d902728183ef8934fd7319"} Sep 07 17:16:03.298571 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.298552 2563 scope.go:117] "RemoveContainer" containerID="0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5" Sep 07 17:16:03.298862 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:16:03.298840 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5\": container with ID starting with 0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5 not found: ID does not exist" containerID="0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5" Sep 07 17:16:03.298942 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.298868 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5"} err="failed to get container status \"0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5\": rpc error: code = NotFound desc = could not find container \"0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5\": container with ID starting with 0a29d58d7f9a906a76f79461882676b9e8f97c070cfe708b061fe78ce8e71db5 not found: ID does not exist" Sep 07 17:16:03.327343 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.327316 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5747c94b8b-7njdw"] Sep 07 17:16:03.330946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:03.330924 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5747c94b8b-7njdw"] Sep 07 17:16:04.426121 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:04.426084 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aba850e-54bb-4030-9af6-066450707767" path="/var/lib/kubelet/pods/1aba850e-54bb-4030-9af6-066450707767/volumes" Sep 07 17:16:08.311603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:08.311525 2563 generic.go:356] "Generic (PLEG): container finished" podID="114e8c2c-8237-416e-a727-815c4faccaf0" containerID="891d3e198510813aa45fcbb63fe7049abbef1ddd435e94e463a7da80d7d0e30d" exitCode=0 Sep 07 17:16:08.311603 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:08.311579 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerDied","Data":"891d3e198510813aa45fcbb63fe7049abbef1ddd435e94e463a7da80d7d0e30d"} Sep 07 17:16:09.318546 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:09.318505 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerStarted","Data":"56c6ddeef05548c537056567a694c8e28f1f341b96995930cdd4268f48f79150"} Sep 07 17:16:14.337213 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:14.337182 2563 generic.go:356] "Generic (PLEG): container finished" podID="114e8c2c-8237-416e-a727-815c4faccaf0" containerID="56c6ddeef05548c537056567a694c8e28f1f341b96995930cdd4268f48f79150" exitCode=0 Sep 07 17:16:14.337588 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:14.337258 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerDied","Data":"56c6ddeef05548c537056567a694c8e28f1f341b96995930cdd4268f48f79150"} Sep 07 17:16:14.350049 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:14.350021 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 07 17:16:15.347461 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:15.347424 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerStarted","Data":"7188c62fe718663a29bc32d1e7c9110d61348bc88e6455cd9bedadf92f65f38b"} Sep 07 17:16:15.347957 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:15.347486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerStarted","Data":"7abf807fb1742cf1ed90347fd100ab05238a7e7794f445fdbe67cf1d4c2c3d55"} Sep 07 17:16:15.347957 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:15.347508 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerStarted","Data":"77d6a8feb6cce8bd69ae8a9c9e35928aa9e8ea9324a1ea5fa387132bf341d5ce"} Sep 07 17:16:15.347957 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:15.347528 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerStarted","Data":"b17c13c5f155fb1123af295b2e05499c70bfeccc5320023255d8147cd3ca98f9"} Sep 07 17:16:15.372466 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:15.372396 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podStartSLOduration=14.012208077 podStartE2EDuration="42.372374268s" totalImagesPullingTime="28.360166191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:16:15.370570376 +0000 UTC m=+483.443471709" watchObservedRunningTime="2026-09-07 17:16:15.372374268 +0000 UTC m=+483.445275602" Sep 07 17:16:16.137163 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.137106 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:16.137374 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.137177 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:16.137374 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.137198 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:16.137374 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.137217 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:16.139242 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.139201 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 07 17:16:16.139456 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.139425 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 07 17:16:16.139561 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.139511 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused" Sep 07 17:16:16.139757 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.139698 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 07 17:16:16.968203 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:16.968165 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:16:19.137621 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:19.137575 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused" Sep 07 17:16:19.138275 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:19.137614 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 07 17:16:19.138457 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:19.137604 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 07 17:16:19.138457 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:19.137617 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 07 17:16:20.878326 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:20.878289 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:16:22.136964 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.136921 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 07 17:16:22.137438 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.136931 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 07 17:16:22.137619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.137592 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:22.138250 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.138225 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:22.138662 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.138642 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:22.230047 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.230013 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:22.389564 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.389471 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:22.392934 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.392909 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:22.625013 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.624949 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:16:22.725492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:22.725388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:16:24.137169 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:24.137135 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:24.137535 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:24.137195 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:25.137781 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:25.137754 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:25.138198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:25.137885 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:25.401772 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:25.401677 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:25.401936 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:25.401864 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:16:25.831653 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:16:25.831614 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:17:27.454253 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.454208 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:17:27.466682 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.466639 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf"] Sep 07 17:17:27.466927 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.466898 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="registry" containerID="cri-o://b17c13c5f155fb1123af295b2e05499c70bfeccc5320023255d8147cd3ca98f9" gracePeriod=30 Sep 07 17:17:27.467014 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.466966 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" containerID="cri-o://7188c62fe718663a29bc32d1e7c9110d61348bc88e6455cd9bedadf92f65f38b" gracePeriod=30 Sep 07 17:17:27.467014 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.466975 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="offline" containerID="cri-o://7abf807fb1742cf1ed90347fd100ab05238a7e7794f445fdbe67cf1d4c2c3d55" gracePeriod=30 Sep 07 17:17:27.467145 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.467015 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" containerID="cri-o://77d6a8feb6cce8bd69ae8a9c9e35928aa9e8ea9324a1ea5fa387132bf341d5ce" gracePeriod=30 Sep 07 17:17:27.902086 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.902032 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85"] Sep 07 17:17:27.902645 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.902582 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aba850e-54bb-4030-9af6-066450707767" containerName="console" Sep 07 17:17:27.902645 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.902608 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aba850e-54bb-4030-9af6-066450707767" containerName="console" Sep 07 17:17:27.902876 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.902701 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aba850e-54bb-4030-9af6-066450707767" containerName="console" Sep 07 17:17:27.907112 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.907090 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:27.910587 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.910381 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-swvwp\"" Sep 07 17:17:27.920554 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.920480 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85"] Sep 07 17:17:27.993292 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.993241 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtmrs\" (UniqueName: \"kubernetes.io/projected/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-kube-api-access-qtmrs\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:27.993498 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.993331 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-offline-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:27.993498 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.993368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:27.993498 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.993397 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:27.993498 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.993421 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-feast-data\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:27.993498 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:27.993452 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.094272 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-offline-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.094325 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.094358 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.094383 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-feast-data\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.094419 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.094459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtmrs\" (UniqueName: \"kubernetes.io/projected/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-kube-api-access-qtmrs\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:17:28.094594 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:17:28.094683 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls podName:8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0 nodeName:}" failed. No retries permitted until 2026-09-07 17:17:28.594658829 +0000 UTC m=+556.667560148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls") pod "feast-simple-feast-setup-7cc9474b77-wfr85" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0") : secret "feast-simple-feast-setup-registry-tls" not found Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:17:28.094931 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:17:28.094982 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls podName:8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0 nodeName:}" failed. No retries permitted until 2026-09-07 17:17:28.594967883 +0000 UTC m=+556.667869198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls") pod "feast-simple-feast-setup-7cc9474b77-wfr85" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0") : secret "feast-simple-feast-setup-ui-tls" not found Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.095001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-feast-data\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:17:28.095022 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 07 17:17:28.095159 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:17:28.095060 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls podName:8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0 nodeName:}" failed. No retries permitted until 2026-09-07 17:17:28.595050336 +0000 UTC m=+556.667951650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls") pod "feast-simple-feast-setup-7cc9474b77-wfr85" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0") : secret "feast-simple-feast-setup-online-tls" not found Sep 07 17:17:28.097604 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.097578 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-offline-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.103901 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.103880 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtmrs\" (UniqueName: \"kubernetes.io/projected/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-kube-api-access-qtmrs\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.599212 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.599163 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.599571 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.599221 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.599571 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.599245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.601673 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.601649 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.601918 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.601897 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.601969 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.601897 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls\") pod \"feast-simple-feast-setup-7cc9474b77-wfr85\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:28.622312 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.622284 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:17:28.637358 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.637323 2563 generic.go:356] "Generic (PLEG): container finished" podID="114e8c2c-8237-416e-a727-815c4faccaf0" containerID="7188c62fe718663a29bc32d1e7c9110d61348bc88e6455cd9bedadf92f65f38b" exitCode=0 Sep 07 17:17:28.637358 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.637353 2563 generic.go:356] "Generic (PLEG): container finished" podID="114e8c2c-8237-416e-a727-815c4faccaf0" containerID="77d6a8feb6cce8bd69ae8a9c9e35928aa9e8ea9324a1ea5fa387132bf341d5ce" exitCode=0 Sep 07 17:17:28.637501 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.637372 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerDied","Data":"7188c62fe718663a29bc32d1e7c9110d61348bc88e6455cd9bedadf92f65f38b"} Sep 07 17:17:28.637501 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.637402 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerDied","Data":"77d6a8feb6cce8bd69ae8a9c9e35928aa9e8ea9324a1ea5fa387132bf341d5ce"} Sep 07 17:17:28.823492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:28.823455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:29.102723 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:29.102698 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85"] Sep 07 17:17:29.104507 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:17:29.104473 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f0f4040_dc86_4e6f_a56f_eee3c46bd0d0.slice/crio-bb3a0e0d687aafcda82ffd8a48f1011fbff019dffc539aa43db3c2cccad5b4c3 WatchSource:0}: Error finding container bb3a0e0d687aafcda82ffd8a48f1011fbff019dffc539aa43db3c2cccad5b4c3: Status 404 returned error can't find the container with id bb3a0e0d687aafcda82ffd8a48f1011fbff019dffc539aa43db3c2cccad5b4c3 Sep 07 17:17:29.643351 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:29.643313 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerStarted","Data":"d85aeabbc610107e4eddba6363bbe04ae96a58ec660aaab090fec483852ef047"} Sep 07 17:17:29.643762 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:29.643361 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerStarted","Data":"bb3a0e0d687aafcda82ffd8a48f1011fbff019dffc539aa43db3c2cccad5b4c3"} Sep 07 17:17:32.390492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:32.390442 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 07 17:17:34.665547 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:34.665514 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerID="d85aeabbc610107e4eddba6363bbe04ae96a58ec660aaab090fec483852ef047" exitCode=0 Sep 07 17:17:34.665991 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:34.665593 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerDied","Data":"d85aeabbc610107e4eddba6363bbe04ae96a58ec660aaab090fec483852ef047"} Sep 07 17:17:35.402293 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:35.402246 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 07 17:17:35.670886 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:35.670806 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerStarted","Data":"56ce665d98f067ad258f011794bcb156cbbf3ac456480231c519daf09ef282e2"} Sep 07 17:17:40.692581 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:40.692543 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerID="56ce665d98f067ad258f011794bcb156cbbf3ac456480231c519daf09ef282e2" exitCode=0 Sep 07 17:17:40.692983 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:40.692623 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerDied","Data":"56ce665d98f067ad258f011794bcb156cbbf3ac456480231c519daf09ef282e2"} Sep 07 17:17:41.703564 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:41.703522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerStarted","Data":"543b4329dc46b26fe516d1a5fe4777a26d7e290acbd21f321d5748617ada579b"} Sep 07 17:17:41.704074 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:41.703591 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerStarted","Data":"5568ce044c8817a49708ae1606ad6952e8663809fa7ee508aa1a4fd037ed1312"} Sep 07 17:17:41.704074 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:41.703615 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerStarted","Data":"2df0ceb41f488f766c10bb15128aabf9abd4ad69be6c59532b2f98f809a28691"} Sep 07 17:17:41.704074 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:41.703634 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerStarted","Data":"eb31fa2f4008eee161e576c1a6b7ce8029b223663cd5cbb5c7d784566be32975"} Sep 07 17:17:41.729860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:41.729790 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podStartSLOduration=14.729767133 podStartE2EDuration="14.729767133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:17:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:17:41.728850597 +0000 UTC m=+569.801751953" watchObservedRunningTime="2026-09-07 17:17:41.729767133 +0000 UTC m=+569.802668471" Sep 07 17:17:42.389660 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:42.389615 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 07 17:17:43.824455 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.824415 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:43.825044 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.825016 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:43.825189 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.825050 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:43.825189 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.825071 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:43.826858 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.826823 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Sep 07 17:17:43.826984 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.826876 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 17:17:43.826984 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.826964 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 17:17:43.827111 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:43.827012 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.25:6571: connect: connection refused" Sep 07 17:17:45.402052 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:45.401997 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 07 17:17:46.824560 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:46.824487 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Sep 07 17:17:46.825097 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:46.824487 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.25:6571: connect: connection refused" Sep 07 17:17:46.825097 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:46.824487 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 17:17:46.825097 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:46.824505 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 17:17:48.823789 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:48.823747 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:48.824188 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:48.823812 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:48.824188 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:48.823826 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:48.824188 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:48.823837 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:49.824521 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:49.824484 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:49.825057 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:49.824819 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:49.825310 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:49.825291 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:49.825404 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:49.825356 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:49.829696 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:49.829674 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:50.749712 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:50.749681 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:50.749898 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:50.749766 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:50.752450 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:50.752428 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:17:51.107708 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:51.107624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:17:52.389979 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:52.389931 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 07 17:17:52.390362 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:52.390071 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:17:55.401380 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:55.401328 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 07 17:17:55.401823 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:55.401509 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:17:57.778862 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:57.778830 2563 generic.go:356] "Generic (PLEG): container finished" podID="114e8c2c-8237-416e-a727-815c4faccaf0" containerID="7abf807fb1742cf1ed90347fd100ab05238a7e7794f445fdbe67cf1d4c2c3d55" exitCode=137 Sep 07 17:17:57.778862 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:57.778854 2563 generic.go:356] "Generic (PLEG): container finished" podID="114e8c2c-8237-416e-a727-815c4faccaf0" containerID="b17c13c5f155fb1123af295b2e05499c70bfeccc5320023255d8147cd3ca98f9" exitCode=137 Sep 07 17:17:57.779254 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:57.778904 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerDied","Data":"7abf807fb1742cf1ed90347fd100ab05238a7e7794f445fdbe67cf1d4c2c3d55"} Sep 07 17:17:57.779254 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:57.778951 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerDied","Data":"b17c13c5f155fb1123af295b2e05499c70bfeccc5320023255d8147cd3ca98f9"} Sep 07 17:17:58.113948 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.113923 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:17:58.176256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.176223 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/114e8c2c-8237-416e-a727-815c4faccaf0-feast-data\") pod \"114e8c2c-8237-416e-a727-815c4faccaf0\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " Sep 07 17:17:58.176256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.176257 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls\") pod \"114e8c2c-8237-416e-a727-815c4faccaf0\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " Sep 07 17:17:58.176485 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.176283 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-offline-tls\") pod \"114e8c2c-8237-416e-a727-815c4faccaf0\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " Sep 07 17:17:58.176485 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.176326 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-ui-tls\") pod \"114e8c2c-8237-416e-a727-815c4faccaf0\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " Sep 07 17:17:58.176485 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.176373 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-online-tls\") pod \"114e8c2c-8237-416e-a727-815c4faccaf0\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " Sep 07 17:17:58.176485 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.176433 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7zqs\" (UniqueName: \"kubernetes.io/projected/114e8c2c-8237-416e-a727-815c4faccaf0-kube-api-access-j7zqs\") pod \"114e8c2c-8237-416e-a727-815c4faccaf0\" (UID: \"114e8c2c-8237-416e-a727-815c4faccaf0\") " Sep 07 17:17:58.176889 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.176862 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114e8c2c-8237-416e-a727-815c4faccaf0-feast-data" (OuterVolumeSpecName: "feast-data") pod "114e8c2c-8237-416e-a727-815c4faccaf0" (UID: "114e8c2c-8237-416e-a727-815c4faccaf0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:17:58.178605 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.178571 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "114e8c2c-8237-416e-a727-815c4faccaf0" (UID: "114e8c2c-8237-416e-a727-815c4faccaf0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:17:58.178605 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.178591 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-online-tls" (OuterVolumeSpecName: "online-tls") pod "114e8c2c-8237-416e-a727-815c4faccaf0" (UID: "114e8c2c-8237-416e-a727-815c4faccaf0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:17:58.178787 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.178643 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114e8c2c-8237-416e-a727-815c4faccaf0-kube-api-access-j7zqs" (OuterVolumeSpecName: "kube-api-access-j7zqs") pod "114e8c2c-8237-416e-a727-815c4faccaf0" (UID: "114e8c2c-8237-416e-a727-815c4faccaf0"). InnerVolumeSpecName "kube-api-access-j7zqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:17:58.178839 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.178829 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "114e8c2c-8237-416e-a727-815c4faccaf0" (UID: "114e8c2c-8237-416e-a727-815c4faccaf0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:17:58.178934 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.178913 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "114e8c2c-8237-416e-a727-815c4faccaf0" (UID: "114e8c2c-8237-416e-a727-815c4faccaf0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:17:58.277436 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.277403 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/114e8c2c-8237-416e-a727-815c4faccaf0-feast-data\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:17:58.277436 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.277430 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-registry-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:17:58.277436 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.277438 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-offline-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:17:58.277436 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.277446 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-ui-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:17:58.277684 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.277453 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/114e8c2c-8237-416e-a727-815c4faccaf0-online-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:17:58.277684 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.277462 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7zqs\" (UniqueName: \"kubernetes.io/projected/114e8c2c-8237-416e-a727-815c4faccaf0-kube-api-access-j7zqs\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:17:58.786537 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.786505 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" Sep 07 17:17:58.786537 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.786516 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf" event={"ID":"114e8c2c-8237-416e-a727-815c4faccaf0","Type":"ContainerDied","Data":"c47516488ec862cc9bf69a9a86941fed3e0138d402221996535c6ab0015a1b30"} Sep 07 17:17:58.787126 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.786579 2563 scope.go:117] "RemoveContainer" containerID="7188c62fe718663a29bc32d1e7c9110d61348bc88e6455cd9bedadf92f65f38b" Sep 07 17:17:58.794861 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.794843 2563 scope.go:117] "RemoveContainer" containerID="7abf807fb1742cf1ed90347fd100ab05238a7e7794f445fdbe67cf1d4c2c3d55" Sep 07 17:17:58.802464 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.802441 2563 scope.go:117] "RemoveContainer" containerID="77d6a8feb6cce8bd69ae8a9c9e35928aa9e8ea9324a1ea5fa387132bf341d5ce" Sep 07 17:17:58.804537 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.804516 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf"] Sep 07 17:17:58.807618 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.807599 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-zfnvf"] Sep 07 17:17:58.810715 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.810696 2563 scope.go:117] "RemoveContainer" containerID="b17c13c5f155fb1123af295b2e05499c70bfeccc5320023255d8147cd3ca98f9" Sep 07 17:17:58.818291 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.818275 2563 scope.go:117] "RemoveContainer" containerID="56c6ddeef05548c537056567a694c8e28f1f341b96995930cdd4268f48f79150" Sep 07 17:17:58.825673 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:17:58.825654 2563 scope.go:117] "RemoveContainer" containerID="891d3e198510813aa45fcbb63fe7049abbef1ddd435e94e463a7da80d7d0e30d" Sep 07 17:18:00.429335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:00.429298 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" path="/var/lib/kubelet/pods/114e8c2c-8237-416e-a727-815c4faccaf0/volumes" Sep 07 17:18:52.325217 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.325126 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:18:52.424979 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.424922 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:18:52.777296 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.777225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:18:52.877317 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.877282 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:18:52.928300 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928268 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99"] Sep 07 17:18:52.928894 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928866 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" Sep 07 17:18:52.928894 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928887 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" Sep 07 17:18:52.928894 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928899 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="feast-init" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928905 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="feast-init" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928914 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928920 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928937 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="feast-apply" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928943 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="feast-apply" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928949 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="registry" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928954 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="registry" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928961 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="offline" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.928966 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="offline" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.929013 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="offline" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.929021 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="online" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.929029 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="ui" Sep 07 17:18:52.929120 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.929037 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="114e8c2c-8237-416e-a727-815c4faccaf0" containerName="registry" Sep 07 17:18:52.932334 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.932317 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:52.934440 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.934419 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 07 17:18:52.934555 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.934491 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 07 17:18:52.934555 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.934491 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 07 17:18:52.934913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.934713 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 07 17:18:52.934913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.934777 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 07 17:18:52.934913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.934807 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-7r69l\"" Sep 07 17:18:52.935104 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.934982 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 07 17:18:52.935311 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.935294 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 07 17:18:52.947620 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:52.947598 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99"] Sep 07 17:18:53.026657 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.026620 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5wxl\" (UniqueName: \"kubernetes.io/projected/182bde20-b604-4499-8c4b-b92e775a603d-kube-api-access-n5wxl\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.026843 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.026667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/182bde20-b604-4499-8c4b-b92e775a603d-registry-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.026843 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.026704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-offline-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.026843 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.026743 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/182bde20-b604-4499-8c4b-b92e775a603d-feast-data\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.026843 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.026812 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.027002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.026841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-ui-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.074465 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.074371 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:18:53.127997 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.127958 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5wxl\" (UniqueName: \"kubernetes.io/projected/182bde20-b604-4499-8c4b-b92e775a603d-kube-api-access-n5wxl\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.128181 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.128001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/182bde20-b604-4499-8c4b-b92e775a603d-registry-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.128181 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.128027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-offline-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.128181 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.128049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/182bde20-b604-4499-8c4b-b92e775a603d-feast-data\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.128181 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.128110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.128181 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.128140 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-ui-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.128445 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:18:53.128261 2563 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 07 17:18:53.128445 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:18:53.128354 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls podName:182bde20-b604-4499-8c4b-b92e775a603d nodeName:}" failed. No retries permitted until 2026-09-07 17:18:53.628331129 +0000 UTC m=+641.701232456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls") pod "feast-simple-feast-remote-setup-987946c98-pgx99" (UID: "182bde20-b604-4499-8c4b-b92e775a603d") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 07 17:18:53.128573 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.128503 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/182bde20-b604-4499-8c4b-b92e775a603d-feast-data\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.128782 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.128762 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/182bde20-b604-4499-8c4b-b92e775a603d-registry-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.130543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.130520 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-offline-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.130705 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.130684 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-ui-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.136269 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.136245 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5wxl\" (UniqueName: \"kubernetes.io/projected/182bde20-b604-4499-8c4b-b92e775a603d-kube-api-access-n5wxl\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.173714 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.173659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:18:53.274032 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.273998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:18:53.633105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.633070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.635346 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.635325 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls\") pod \"feast-simple-feast-remote-setup-987946c98-pgx99\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:53.843193 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:53.843152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:18:54.123753 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:54.123705 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99"] Sep 07 17:18:54.125486 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:18:54.125457 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod182bde20_b604_4499_8c4b_b92e775a603d.slice/crio-a43583d526dfefdde82e1aed6c233d7c41f6c51fe0f97c94e5f73d4b57fede2a WatchSource:0}: Error finding container a43583d526dfefdde82e1aed6c233d7c41f6c51fe0f97c94e5f73d4b57fede2a: Status 404 returned error can't find the container with id a43583d526dfefdde82e1aed6c233d7c41f6c51fe0f97c94e5f73d4b57fede2a Sep 07 17:18:55.001130 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:55.001093 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerStarted","Data":"01d1e76fff90a7f507cfa126a781974cf5d284ccdc15136145c90b877431ac23"} Sep 07 17:18:55.001130 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:55.001136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerStarted","Data":"a43583d526dfefdde82e1aed6c233d7c41f6c51fe0f97c94e5f73d4b57fede2a"} Sep 07 17:18:59.019792 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:59.019752 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerDied","Data":"01d1e76fff90a7f507cfa126a781974cf5d284ccdc15136145c90b877431ac23"} Sep 07 17:18:59.020296 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:18:59.019749 2563 generic.go:356] "Generic (PLEG): container finished" podID="182bde20-b604-4499-8c4b-b92e775a603d" containerID="01d1e76fff90a7f507cfa126a781974cf5d284ccdc15136145c90b877431ac23" exitCode=0 Sep 07 17:19:00.026833 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:00.026798 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerStarted","Data":"44806b6785967769a1f7cd8840bf66ccf9f08e8b976a5de4c3241ab9328a391d"} Sep 07 17:19:24.120514 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:24.120484 2563 generic.go:356] "Generic (PLEG): container finished" podID="182bde20-b604-4499-8c4b-b92e775a603d" containerID="44806b6785967769a1f7cd8840bf66ccf9f08e8b976a5de4c3241ab9328a391d" exitCode=0 Sep 07 17:19:24.120901 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:24.120559 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerDied","Data":"44806b6785967769a1f7cd8840bf66ccf9f08e8b976a5de4c3241ab9328a391d"} Sep 07 17:19:24.125153 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:24.125132 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 07 17:19:25.129455 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:25.129411 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerStarted","Data":"b8836f507d25a29c53ad2efc56b97de2db0bb9a68ebce95d937690fb53755b32"} Sep 07 17:19:25.129938 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:25.129471 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerStarted","Data":"24e792e7a81a1ea3804e95f4fa6bfc52f617850a2fd0a0d46767799878f91a1f"} Sep 07 17:19:25.129938 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:25.129486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerStarted","Data":"13b2fb856c0a16e042455145758e48a9900e69cd6cd455de088b307342a1f890"} Sep 07 17:19:25.156249 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:25.156184 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podStartSLOduration=33.156164261 podStartE2EDuration="33.156164261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:18:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:19:25.155524 +0000 UTC m=+673.228425358" watchObservedRunningTime="2026-09-07 17:19:25.156164261 +0000 UTC m=+673.229065594" Sep 07 17:19:26.844319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:26.844273 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:26.844702 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:26.844346 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:26.844702 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:26.844367 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:26.846216 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:26.846182 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 07 17:19:26.846351 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:26.846211 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 17:19:26.846351 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:26.846274 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 17:19:29.844427 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:29.844380 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 07 17:19:29.844914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:29.844390 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 17:19:29.844914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:29.844584 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 17:19:32.844039 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:32.844001 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:32.844473 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:32.844284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:32.844526 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:32.844482 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:32.844825 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:32.844810 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:32.869183 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:32.869160 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:33.166533 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:33.166450 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:33.166533 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:33.166494 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:33.187632 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:33.187605 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:34.170501 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:34.170471 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:19:34.486851 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:19:34.486751 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:20:37.195787 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.195749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:20:37.197571 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.197544 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99"] Sep 07 17:20:37.197889 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.197857 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" containerID="cri-o://13b2fb856c0a16e042455145758e48a9900e69cd6cd455de088b307342a1f890" gracePeriod=30 Sep 07 17:20:37.198017 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.197922 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="offline" containerID="cri-o://24e792e7a81a1ea3804e95f4fa6bfc52f617850a2fd0a0d46767799878f91a1f" gracePeriod=30 Sep 07 17:20:37.198122 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.198089 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" containerID="cri-o://b8836f507d25a29c53ad2efc56b97de2db0bb9a68ebce95d937690fb53755b32" gracePeriod=30 Sep 07 17:20:37.290877 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.290826 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:20:37.313671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.313630 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85"] Sep 07 17:20:37.314054 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.313932 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="registry" containerID="cri-o://eb31fa2f4008eee161e576c1a6b7ce8029b223663cd5cbb5c7d784566be32975" gracePeriod=30 Sep 07 17:20:37.314054 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.313984 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" containerID="cri-o://2df0ceb41f488f766c10bb15128aabf9abd4ad69be6c59532b2f98f809a28691" gracePeriod=30 Sep 07 17:20:37.314054 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.313960 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" containerID="cri-o://543b4329dc46b26fe516d1a5fe4777a26d7e290acbd21f321d5748617ada579b" gracePeriod=30 Sep 07 17:20:37.314290 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.314021 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="offline" containerID="cri-o://5568ce044c8817a49708ae1606ad6952e8663809fa7ee508aa1a4fd037ed1312" gracePeriod=30 Sep 07 17:20:37.390869 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.390828 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:20:37.464440 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:37.464357 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:20:38.423345 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.423259 2563 generic.go:356] "Generic (PLEG): container finished" podID="182bde20-b604-4499-8c4b-b92e775a603d" containerID="b8836f507d25a29c53ad2efc56b97de2db0bb9a68ebce95d937690fb53755b32" exitCode=0 Sep 07 17:20:38.423345 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.423284 2563 generic.go:356] "Generic (PLEG): container finished" podID="182bde20-b604-4499-8c4b-b92e775a603d" containerID="13b2fb856c0a16e042455145758e48a9900e69cd6cd455de088b307342a1f890" exitCode=0 Sep 07 17:20:38.426465 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.426427 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerDied","Data":"b8836f507d25a29c53ad2efc56b97de2db0bb9a68ebce95d937690fb53755b32"} Sep 07 17:20:38.426611 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.426490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerDied","Data":"13b2fb856c0a16e042455145758e48a9900e69cd6cd455de088b307342a1f890"} Sep 07 17:20:38.431678 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.431655 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerID="543b4329dc46b26fe516d1a5fe4777a26d7e290acbd21f321d5748617ada579b" exitCode=0 Sep 07 17:20:38.431678 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.431676 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerID="2df0ceb41f488f766c10bb15128aabf9abd4ad69be6c59532b2f98f809a28691" exitCode=0 Sep 07 17:20:38.431848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.431717 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerDied","Data":"543b4329dc46b26fe516d1a5fe4777a26d7e290acbd21f321d5748617ada579b"} Sep 07 17:20:38.431848 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:38.431764 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerDied","Data":"2df0ceb41f488f766c10bb15128aabf9abd4ad69be6c59532b2f98f809a28691"} Sep 07 17:20:40.749588 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:40.749542 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 17:20:40.750065 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:40.749542 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 17:20:42.471596 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:42.471558 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:20:42.845078 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:42.845033 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 17:20:43.167183 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:43.167093 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 17:20:50.750024 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:50.749965 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 17:20:50.750502 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:50.749984 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 17:20:52.845288 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:52.845236 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 17:20:53.166860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:20:53.166766 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 17:21:00.749965 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:00.749928 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 17:21:00.750370 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:00.749922 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 17:21:00.750370 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:00.750059 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:21:00.750370 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:00.750125 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:21:02.845324 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:02.845285 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 17:21:02.845715 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:02.845418 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:21:03.166926 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:03.166839 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 17:21:03.167071 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:03.166958 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:21:06.479629 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:06.479582 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:07.546396 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.546366 2563 generic.go:356] "Generic (PLEG): container finished" podID="182bde20-b604-4499-8c4b-b92e775a603d" containerID="24e792e7a81a1ea3804e95f4fa6bfc52f617850a2fd0a0d46767799878f91a1f" exitCode=137 Sep 07 17:21:07.546834 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.546445 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerDied","Data":"24e792e7a81a1ea3804e95f4fa6bfc52f617850a2fd0a0d46767799878f91a1f"} Sep 07 17:21:07.549398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.549368 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerID="5568ce044c8817a49708ae1606ad6952e8663809fa7ee508aa1a4fd037ed1312" exitCode=137 Sep 07 17:21:07.549398 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.549395 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerID="eb31fa2f4008eee161e576c1a6b7ce8029b223663cd5cbb5c7d784566be32975" exitCode=137 Sep 07 17:21:07.549530 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.549439 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerDied","Data":"5568ce044c8817a49708ae1606ad6952e8663809fa7ee508aa1a4fd037ed1312"} Sep 07 17:21:07.549530 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.549473 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerDied","Data":"eb31fa2f4008eee161e576c1a6b7ce8029b223663cd5cbb5c7d784566be32975"} Sep 07 17:21:07.839047 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.839020 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:21:07.948171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.948148 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:21:07.979679 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.979650 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-offline-tls\") pod \"182bde20-b604-4499-8c4b-b92e775a603d\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " Sep 07 17:21:07.979870 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.979697 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls\") pod \"182bde20-b604-4499-8c4b-b92e775a603d\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " Sep 07 17:21:07.979870 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.979752 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/182bde20-b604-4499-8c4b-b92e775a603d-registry-tls\") pod \"182bde20-b604-4499-8c4b-b92e775a603d\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " Sep 07 17:21:07.979870 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.979770 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-ui-tls\") pod \"182bde20-b604-4499-8c4b-b92e775a603d\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " Sep 07 17:21:07.979870 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.979825 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/182bde20-b604-4499-8c4b-b92e775a603d-feast-data\") pod \"182bde20-b604-4499-8c4b-b92e775a603d\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " Sep 07 17:21:07.979870 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.979870 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5wxl\" (UniqueName: \"kubernetes.io/projected/182bde20-b604-4499-8c4b-b92e775a603d-kube-api-access-n5wxl\") pod \"182bde20-b604-4499-8c4b-b92e775a603d\" (UID: \"182bde20-b604-4499-8c4b-b92e775a603d\") " Sep 07 17:21:07.980156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.980125 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/182bde20-b604-4499-8c4b-b92e775a603d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "182bde20-b604-4499-8c4b-b92e775a603d" (UID: "182bde20-b604-4499-8c4b-b92e775a603d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 17:21:07.980440 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.980391 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182bde20-b604-4499-8c4b-b92e775a603d-feast-data" (OuterVolumeSpecName: "feast-data") pod "182bde20-b604-4499-8c4b-b92e775a603d" (UID: "182bde20-b604-4499-8c4b-b92e775a603d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:21:07.982054 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.982031 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "182bde20-b604-4499-8c4b-b92e775a603d" (UID: "182bde20-b604-4499-8c4b-b92e775a603d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:21:07.982202 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.982185 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "182bde20-b604-4499-8c4b-b92e775a603d" (UID: "182bde20-b604-4499-8c4b-b92e775a603d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:21:07.982279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.982253 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls" (OuterVolumeSpecName: "online-tls") pod "182bde20-b604-4499-8c4b-b92e775a603d" (UID: "182bde20-b604-4499-8c4b-b92e775a603d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:21:07.982336 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:07.982294 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182bde20-b604-4499-8c4b-b92e775a603d-kube-api-access-n5wxl" (OuterVolumeSpecName: "kube-api-access-n5wxl") pod "182bde20-b604-4499-8c4b-b92e775a603d" (UID: "182bde20-b604-4499-8c4b-b92e775a603d"). InnerVolumeSpecName "kube-api-access-n5wxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:21:08.080553 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080452 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls\") pod \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " Sep 07 17:21:08.080553 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080493 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-offline-tls\") pod \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " Sep 07 17:21:08.080841 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080574 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls\") pod \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " Sep 07 17:21:08.080841 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080609 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtmrs\" (UniqueName: \"kubernetes.io/projected/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-kube-api-access-qtmrs\") pod \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " Sep 07 17:21:08.080841 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080657 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls\") pod \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " Sep 07 17:21:08.080841 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080680 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-feast-data\") pod \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\" (UID: \"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0\") " Sep 07 17:21:08.081040 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080916 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/182bde20-b604-4499-8c4b-b92e775a603d-feast-data\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.081040 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080935 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5wxl\" (UniqueName: \"kubernetes.io/projected/182bde20-b604-4499-8c4b-b92e775a603d-kube-api-access-n5wxl\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.081040 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080948 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-offline-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.081040 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080962 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-online-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.081040 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080975 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/182bde20-b604-4499-8c4b-b92e775a603d-registry-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.081040 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.080987 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/182bde20-b604-4499-8c4b-b92e775a603d-ui-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.081351 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.081323 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-feast-data" (OuterVolumeSpecName: "feast-data") pod "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:21:08.082650 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.082623 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:21:08.082784 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.082748 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-kube-api-access-qtmrs" (OuterVolumeSpecName: "kube-api-access-qtmrs") pod "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0"). InnerVolumeSpecName "kube-api-access-qtmrs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:21:08.082837 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.082790 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:21:08.082989 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.082975 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls" (OuterVolumeSpecName: "online-tls") pod "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:21:08.083056 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.083043 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" (UID: "8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:21:08.182058 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.182025 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-online-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.182058 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.182055 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtmrs\" (UniqueName: \"kubernetes.io/projected/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-kube-api-access-qtmrs\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.182214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.182064 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-ui-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.182214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.182073 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-feast-data\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.182214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.182082 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-registry-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.182214 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.182089 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0-offline-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:21:08.556413 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.556374 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" event={"ID":"182bde20-b604-4499-8c4b-b92e775a603d","Type":"ContainerDied","Data":"a43583d526dfefdde82e1aed6c233d7c41f6c51fe0f97c94e5f73d4b57fede2a"} Sep 07 17:21:08.556413 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.556392 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99" Sep 07 17:21:08.556963 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.556437 2563 scope.go:117] "RemoveContainer" containerID="b8836f507d25a29c53ad2efc56b97de2db0bb9a68ebce95d937690fb53755b32" Sep 07 17:21:08.559773 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.559706 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" event={"ID":"8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0","Type":"ContainerDied","Data":"bb3a0e0d687aafcda82ffd8a48f1011fbff019dffc539aa43db3c2cccad5b4c3"} Sep 07 17:21:08.559773 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.559763 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85" Sep 07 17:21:08.565933 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.565916 2563 scope.go:117] "RemoveContainer" containerID="24e792e7a81a1ea3804e95f4fa6bfc52f617850a2fd0a0d46767799878f91a1f" Sep 07 17:21:08.573350 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.573318 2563 scope.go:117] "RemoveContainer" containerID="13b2fb856c0a16e042455145758e48a9900e69cd6cd455de088b307342a1f890" Sep 07 17:21:08.574583 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.574569 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99"] Sep 07 17:21:08.577617 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.577593 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-987946c98-pgx99"] Sep 07 17:21:08.580988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.580970 2563 scope.go:117] "RemoveContainer" containerID="44806b6785967769a1f7cd8840bf66ccf9f08e8b976a5de4c3241ab9328a391d" Sep 07 17:21:08.587933 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.587911 2563 scope.go:117] "RemoveContainer" containerID="01d1e76fff90a7f507cfa126a781974cf5d284ccdc15136145c90b877431ac23" Sep 07 17:21:08.591462 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.591443 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85"] Sep 07 17:21:08.595206 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.595187 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7cc9474b77-wfr85"] Sep 07 17:21:08.596061 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.596042 2563 scope.go:117] "RemoveContainer" containerID="543b4329dc46b26fe516d1a5fe4777a26d7e290acbd21f321d5748617ada579b" Sep 07 17:21:08.602595 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.602580 2563 scope.go:117] "RemoveContainer" containerID="5568ce044c8817a49708ae1606ad6952e8663809fa7ee508aa1a4fd037ed1312" Sep 07 17:21:08.609516 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.609498 2563 scope.go:117] "RemoveContainer" containerID="2df0ceb41f488f766c10bb15128aabf9abd4ad69be6c59532b2f98f809a28691" Sep 07 17:21:08.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.615932 2563 scope.go:117] "RemoveContainer" containerID="eb31fa2f4008eee161e576c1a6b7ce8029b223663cd5cbb5c7d784566be32975" Sep 07 17:21:08.622718 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.622703 2563 scope.go:117] "RemoveContainer" containerID="56ce665d98f067ad258f011794bcb156cbbf3ac456480231c519daf09ef282e2" Sep 07 17:21:08.629410 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:08.629394 2563 scope.go:117] "RemoveContainer" containerID="d85aeabbc610107e4eddba6363bbe04ae96a58ec660aaab090fec483852ef047" Sep 07 17:21:10.425194 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:10.425159 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="182bde20-b604-4499-8c4b-b92e775a603d" path="/var/lib/kubelet/pods/182bde20-b604-4499-8c4b-b92e775a603d/volumes" Sep 07 17:21:10.425931 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:10.425908 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" path="/var/lib/kubelet/pods/8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0/volumes" Sep 07 17:21:20.311477 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.311445 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:20.522027 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.521993 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-fgptp"] Sep 07 17:21:20.522427 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522408 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="feast-init" Sep 07 17:21:20.522472 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522432 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="feast-init" Sep 07 17:21:20.522472 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522449 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="feast-apply" Sep 07 17:21:20.522472 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522459 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="feast-apply" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522474 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522485 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522497 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="feast-init" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522506 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="feast-init" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522516 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522526 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522540 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="registry" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522550 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="registry" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522562 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" Sep 07 17:21:20.522580 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522571 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522586 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="feast-apply" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522596 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="feast-apply" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522607 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522615 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522627 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="offline" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522636 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="offline" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522651 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="offline" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522660 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="offline" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522747 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="ui" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522760 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="offline" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522767 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="online" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522776 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="registry" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522783 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="182bde20-b604-4499-8c4b-b92e775a603d" containerName="ui" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522791 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="online" Sep 07 17:21:20.522897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.522799 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0f4040-dc86-4e6f-a56f-eee3c46bd0d0" containerName="offline" Sep 07 17:21:20.528970 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.528950 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.530860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.530841 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-ptm9r\"" Sep 07 17:21:20.531472 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.531450 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 17:21:20.531557 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.531452 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 07 17:21:20.531557 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.531503 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 17:21:20.533466 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.533442 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-fgptp"] Sep 07 17:21:20.553837 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.553810 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-6ljtn"] Sep 07 17:21:20.557197 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.557176 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-6ljtn" Sep 07 17:21:20.564872 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.564846 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-6ljtn"] Sep 07 17:21:20.580034 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.580009 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhksz\" (UniqueName: \"kubernetes.io/projected/8435f5ee-a079-46cd-8a0f-35bb12ef06b3-kube-api-access-dhksz\") pod \"redis-69bbd798b6-6ljtn\" (UID: \"8435f5ee-a079-46cd-8a0f-35bb12ef06b3\") " pod="test-ns-feast/redis-69bbd798b6-6ljtn" Sep 07 17:21:20.580191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.580046 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/44631e6c-6cc4-40d6-b6f9-c113101efd3f-postgresdata\") pod \"postgres-864846d4f5-fgptp\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.580191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.580064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmfmb\" (UniqueName: \"kubernetes.io/projected/44631e6c-6cc4-40d6-b6f9-c113101efd3f-kube-api-access-mmfmb\") pod \"postgres-864846d4f5-fgptp\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.583068 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.583039 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:20.680879 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.680847 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhksz\" (UniqueName: \"kubernetes.io/projected/8435f5ee-a079-46cd-8a0f-35bb12ef06b3-kube-api-access-dhksz\") pod \"redis-69bbd798b6-6ljtn\" (UID: \"8435f5ee-a079-46cd-8a0f-35bb12ef06b3\") " pod="test-ns-feast/redis-69bbd798b6-6ljtn" Sep 07 17:21:20.681041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.680888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/44631e6c-6cc4-40d6-b6f9-c113101efd3f-postgresdata\") pod \"postgres-864846d4f5-fgptp\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.681041 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.680910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmfmb\" (UniqueName: \"kubernetes.io/projected/44631e6c-6cc4-40d6-b6f9-c113101efd3f-kube-api-access-mmfmb\") pod \"postgres-864846d4f5-fgptp\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.681258 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.681242 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/44631e6c-6cc4-40d6-b6f9-c113101efd3f-postgresdata\") pod \"postgres-864846d4f5-fgptp\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.691375 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.691351 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhksz\" (UniqueName: \"kubernetes.io/projected/8435f5ee-a079-46cd-8a0f-35bb12ef06b3-kube-api-access-dhksz\") pod \"redis-69bbd798b6-6ljtn\" (UID: \"8435f5ee-a079-46cd-8a0f-35bb12ef06b3\") " pod="test-ns-feast/redis-69bbd798b6-6ljtn" Sep 07 17:21:20.691493 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.691384 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmfmb\" (UniqueName: \"kubernetes.io/projected/44631e6c-6cc4-40d6-b6f9-c113101efd3f-kube-api-access-mmfmb\") pod \"postgres-864846d4f5-fgptp\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.754486 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.754456 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:20.850795 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.850706 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:21:20.866825 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:20.866804 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-6ljtn" Sep 07 17:21:21.120502 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:21.120365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-fgptp"] Sep 07 17:21:21.123332 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:21:21.123292 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44631e6c_6cc4_40d6_b6f9_c113101efd3f.slice/crio-d2c507b8eb90ea7ea18a1798dd9600ca713c982e18efec3bc1c238fe480d91b2 WatchSource:0}: Error finding container d2c507b8eb90ea7ea18a1798dd9600ca713c982e18efec3bc1c238fe480d91b2: Status 404 returned error can't find the container with id d2c507b8eb90ea7ea18a1798dd9600ca713c982e18efec3bc1c238fe480d91b2 Sep 07 17:21:21.125112 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:21.125095 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 17:21:21.141222 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:21.141199 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-6ljtn"] Sep 07 17:21:21.143117 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:21:21.143095 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8435f5ee_a079_46cd_8a0f_35bb12ef06b3.slice/crio-0ffc1e8a0cff1f427180e3af93b52f007cde7a036e1793548e42d903286b8c88 WatchSource:0}: Error finding container 0ffc1e8a0cff1f427180e3af93b52f007cde7a036e1793548e42d903286b8c88: Status 404 returned error can't find the container with id 0ffc1e8a0cff1f427180e3af93b52f007cde7a036e1793548e42d903286b8c88 Sep 07 17:21:21.612543 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:21.612506 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-6ljtn" event={"ID":"8435f5ee-a079-46cd-8a0f-35bb12ef06b3","Type":"ContainerStarted","Data":"0ffc1e8a0cff1f427180e3af93b52f007cde7a036e1793548e42d903286b8c88"} Sep 07 17:21:21.613566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:21.613544 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-fgptp" event={"ID":"44631e6c-6cc4-40d6-b6f9-c113101efd3f","Type":"ContainerStarted","Data":"d2c507b8eb90ea7ea18a1798dd9600ca713c982e18efec3bc1c238fe480d91b2"} Sep 07 17:21:23.622205 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:23.622168 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-6ljtn" event={"ID":"8435f5ee-a079-46cd-8a0f-35bb12ef06b3","Type":"ContainerStarted","Data":"be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa"} Sep 07 17:21:23.639409 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:23.638973 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-6ljtn" podStartSLOduration=2.026125434 podStartE2EDuration="3.638951928s" totalImagesPullingTime="1.612826494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:21:23.637574486 +0000 UTC m=+791.710475819" watchObservedRunningTime="2026-09-07 17:21:23.638951928 +0000 UTC m=+791.711853262" Sep 07 17:21:24.634545 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:24.634491 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:26.635697 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:26.635659 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-fgptp" event={"ID":"44631e6c-6cc4-40d6-b6f9-c113101efd3f","Type":"ContainerStarted","Data":"2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63"} Sep 07 17:21:26.660025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:26.659961 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-fgptp" podStartSLOduration=1.690123563 podStartE2EDuration="6.659938846s" totalImagesPullingTime="4.969815283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:21:26.658211696 +0000 UTC m=+794.731113029" watchObservedRunningTime="2026-09-07 17:21:26.659938846 +0000 UTC m=+794.732840184" Sep 07 17:21:31.251283 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.251246 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:31.349537 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.349495 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:31.387378 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.387339 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-776f6db575-kbcw6"] Sep 07 17:21:31.394485 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.394459 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.396640 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.396614 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 07 17:21:31.404891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.396889 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 07 17:21:31.404891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.396692 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 07 17:21:31.404891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.397574 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 07 17:21:31.404891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.398763 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-mlc5l\"" Sep 07 17:21:31.404891 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.404186 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-776f6db575-kbcw6"] Sep 07 17:21:31.470166 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.470128 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sdml\" (UniqueName: \"kubernetes.io/projected/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-kube-api-access-6sdml\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.470335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.470175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-offline-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.470335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.470192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.470335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.470221 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-online-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.470335 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.470263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-feast-data\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.503887 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.503807 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:31.570684 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.570645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sdml\" (UniqueName: \"kubernetes.io/projected/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-kube-api-access-6sdml\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.570913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.570698 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-offline-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.570913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.570788 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.570913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.570837 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-online-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.570913 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:21:31.570898 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 07 17:21:31.571133 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:21:31.570957 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls podName:5e10f25e-5903-4fc6-adc3-0eeb4fe17c90 nodeName:}" failed. No retries permitted until 2026-09-07 17:21:32.070940654 +0000 UTC m=+800.143841987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls") pod "feast-credit-scoring-776f6db575-kbcw6" (UID: "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90") : secret "feast-credit-scoring-registry-tls" not found Sep 07 17:21:31.571133 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.571035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-feast-data\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.571400 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.571379 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-feast-data\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.573131 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.573110 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-offline-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.573264 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.573247 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-online-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.579323 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.579301 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sdml\" (UniqueName: \"kubernetes.io/projected/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-kube-api-access-6sdml\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:31.661786 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.661720 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:31.762566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:31.762472 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:21:32.075430 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:32.075335 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:32.077812 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:32.077780 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls\") pod \"feast-credit-scoring-776f6db575-kbcw6\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:32.312163 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:32.312125 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:32.600545 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:32.600513 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-776f6db575-kbcw6"] Sep 07 17:21:32.603928 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:21:32.603900 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e10f25e_5903_4fc6_adc3_0eeb4fe17c90.slice/crio-1f1be51c899a42326cfc97b85c7a22d0865a4561688e96ac95d3ebe1c5b70a6c WatchSource:0}: Error finding container 1f1be51c899a42326cfc97b85c7a22d0865a4561688e96ac95d3ebe1c5b70a6c: Status 404 returned error can't find the container with id 1f1be51c899a42326cfc97b85c7a22d0865a4561688e96ac95d3ebe1c5b70a6c Sep 07 17:21:32.659418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:32.659386 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerStarted","Data":"1f1be51c899a42326cfc97b85c7a22d0865a4561688e96ac95d3ebe1c5b70a6c"} Sep 07 17:21:33.665874 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:33.665837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerStarted","Data":"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a"} Sep 07 17:21:34.671476 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:34.671387 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerID="4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a" exitCode=0 Sep 07 17:21:34.671859 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:34.671471 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerDied","Data":"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a"} Sep 07 17:21:35.677252 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:35.677217 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerStarted","Data":"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1"} Sep 07 17:21:41.704879 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:41.704792 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerID="273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1" exitCode=0 Sep 07 17:21:41.704879 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:41.704868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerDied","Data":"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1"} Sep 07 17:21:42.713857 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:42.713812 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerStarted","Data":"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce"} Sep 07 17:21:42.714330 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:42.713889 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerStarted","Data":"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4"} Sep 07 17:21:42.714330 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:42.713919 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerStarted","Data":"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2"} Sep 07 17:21:42.746030 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:42.745970 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podStartSLOduration=11.745948221999999 podStartE2EDuration="11.745948222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:21:42.745277372 +0000 UTC m=+810.818178743" watchObservedRunningTime="2026-09-07 17:21:42.745948222 +0000 UTC m=+810.818849556" Sep 07 17:21:44.312708 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:44.312665 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:44.313148 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:44.312771 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:44.313148 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:44.312801 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:44.314704 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:44.314670 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 07 17:21:44.314857 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:44.314696 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 07 17:21:44.314857 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:44.314748 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 17:21:47.313046 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:47.313001 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 07 17:21:47.313046 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:47.313028 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 17:21:47.313606 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:47.313007 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 07 17:21:50.312923 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:50.312870 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 07 17:21:50.313886 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:50.313860 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:50.314313 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:50.314292 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:50.314810 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:50.314793 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:50.321641 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:50.321626 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:50.759692 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:50.759663 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:50.764391 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:50.764368 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:52.312831 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:52.312792 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:53.312972 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:53.312922 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 07 17:21:56.313942 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:56.313910 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:56.314672 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:56.314649 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:21:56.655832 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:21:56.655752 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:22:54.476783 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:54.476746 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:22:59.452530 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.452496 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-776f6db575-kbcw6"] Sep 07 17:22:59.452959 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.452754 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" containerID="cri-o://f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2" gracePeriod=30 Sep 07 17:22:59.452959 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.452863 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="offline" containerID="cri-o://6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce" gracePeriod=30 Sep 07 17:22:59.453068 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.452857 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" containerID="cri-o://f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4" gracePeriod=30 Sep 07 17:22:59.467425 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.467398 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-fgptp"] Sep 07 17:22:59.467577 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.467550 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-fgptp" podUID="44631e6c-6cc4-40d6-b6f9-c113101efd3f" containerName="postgres" containerID="cri-o://2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63" gracePeriod=30 Sep 07 17:22:59.471883 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.471863 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-6ljtn"] Sep 07 17:22:59.475695 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.472009 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-6ljtn" podUID="8435f5ee-a079-46cd-8a0f-35bb12ef06b3" containerName="redis" containerID="cri-o://be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa" gracePeriod=30 Sep 07 17:22:59.527489 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.527460 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:22:59.723544 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.723522 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-6ljtn" Sep 07 17:22:59.741414 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.741382 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:22:59.920513 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.920478 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/44631e6c-6cc4-40d6-b6f9-c113101efd3f-postgresdata\") pod \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " Sep 07 17:22:59.920513 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.920514 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmfmb\" (UniqueName: \"kubernetes.io/projected/44631e6c-6cc4-40d6-b6f9-c113101efd3f-kube-api-access-mmfmb\") pod \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\" (UID: \"44631e6c-6cc4-40d6-b6f9-c113101efd3f\") " Sep 07 17:22:59.920828 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.920544 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhksz\" (UniqueName: \"kubernetes.io/projected/8435f5ee-a079-46cd-8a0f-35bb12ef06b3-kube-api-access-dhksz\") pod \"8435f5ee-a079-46cd-8a0f-35bb12ef06b3\" (UID: \"8435f5ee-a079-46cd-8a0f-35bb12ef06b3\") " Sep 07 17:22:59.922636 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.922601 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44631e6c-6cc4-40d6-b6f9-c113101efd3f-kube-api-access-mmfmb" (OuterVolumeSpecName: "kube-api-access-mmfmb") pod "44631e6c-6cc4-40d6-b6f9-c113101efd3f" (UID: "44631e6c-6cc4-40d6-b6f9-c113101efd3f"). InnerVolumeSpecName "kube-api-access-mmfmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:22:59.922636 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.922619 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8435f5ee-a079-46cd-8a0f-35bb12ef06b3-kube-api-access-dhksz" (OuterVolumeSpecName: "kube-api-access-dhksz") pod "8435f5ee-a079-46cd-8a0f-35bb12ef06b3" (UID: "8435f5ee-a079-46cd-8a0f-35bb12ef06b3"). InnerVolumeSpecName "kube-api-access-dhksz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:22:59.956288 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:22:59.956249 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44631e6c-6cc4-40d6-b6f9-c113101efd3f-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "44631e6c-6cc4-40d6-b6f9-c113101efd3f" (UID: "44631e6c-6cc4-40d6-b6f9-c113101efd3f"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:23:00.021944 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.021898 2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/44631e6c-6cc4-40d6-b6f9-c113101efd3f-postgresdata\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:00.021944 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.021943 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmfmb\" (UniqueName: \"kubernetes.io/projected/44631e6c-6cc4-40d6-b6f9-c113101efd3f-kube-api-access-mmfmb\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:00.021944 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.021953 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhksz\" (UniqueName: \"kubernetes.io/projected/8435f5ee-a079-46cd-8a0f-35bb12ef06b3-kube-api-access-dhksz\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:00.043743 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.043696 2563 generic.go:356] "Generic (PLEG): container finished" podID="8435f5ee-a079-46cd-8a0f-35bb12ef06b3" containerID="be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa" exitCode=0 Sep 07 17:23:00.043925 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.043759 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-6ljtn" Sep 07 17:23:00.043925 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.043785 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-6ljtn" event={"ID":"8435f5ee-a079-46cd-8a0f-35bb12ef06b3","Type":"ContainerDied","Data":"be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa"} Sep 07 17:23:00.043925 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.043831 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-6ljtn" event={"ID":"8435f5ee-a079-46cd-8a0f-35bb12ef06b3","Type":"ContainerDied","Data":"0ffc1e8a0cff1f427180e3af93b52f007cde7a036e1793548e42d903286b8c88"} Sep 07 17:23:00.043925 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.043855 2563 scope.go:117] "RemoveContainer" containerID="be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa" Sep 07 17:23:00.045239 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.045219 2563 generic.go:356] "Generic (PLEG): container finished" podID="44631e6c-6cc4-40d6-b6f9-c113101efd3f" containerID="2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63" exitCode=0 Sep 07 17:23:00.045342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.045270 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-fgptp" Sep 07 17:23:00.045342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.045302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-fgptp" event={"ID":"44631e6c-6cc4-40d6-b6f9-c113101efd3f","Type":"ContainerDied","Data":"2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63"} Sep 07 17:23:00.045342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.045332 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-fgptp" event={"ID":"44631e6c-6cc4-40d6-b6f9-c113101efd3f","Type":"ContainerDied","Data":"d2c507b8eb90ea7ea18a1798dd9600ca713c982e18efec3bc1c238fe480d91b2"} Sep 07 17:23:00.054499 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.054476 2563 scope.go:117] "RemoveContainer" containerID="be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa" Sep 07 17:23:00.054813 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:23:00.054792 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa\": container with ID starting with be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa not found: ID does not exist" containerID="be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa" Sep 07 17:23:00.054914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.054819 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa"} err="failed to get container status \"be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa\": rpc error: code = NotFound desc = could not find container \"be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa\": container with ID starting with be8c23da2f6d1fbfdc2db46762284571ea776dd7e74f1437e39feabde7dcf6fa not found: ID does not exist" Sep 07 17:23:00.054914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.054837 2563 scope.go:117] "RemoveContainer" containerID="2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63" Sep 07 17:23:00.062550 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.062532 2563 scope.go:117] "RemoveContainer" containerID="2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63" Sep 07 17:23:00.062822 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:23:00.062801 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63\": container with ID starting with 2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63 not found: ID does not exist" containerID="2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63" Sep 07 17:23:00.062913 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.062828 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63"} err="failed to get container status \"2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63\": rpc error: code = NotFound desc = could not find container \"2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63\": container with ID starting with 2147d47aabe9b32d08d575591de644dcb00ec352062394944037485d984f8b63 not found: ID does not exist" Sep 07 17:23:00.066561 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.066539 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-6ljtn"] Sep 07 17:23:00.068190 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.068170 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-6ljtn"] Sep 07 17:23:00.077070 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.077048 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-fgptp"] Sep 07 17:23:00.082858 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.082832 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-fgptp"] Sep 07 17:23:00.428288 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.428252 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44631e6c-6cc4-40d6-b6f9-c113101efd3f" path="/var/lib/kubelet/pods/44631e6c-6cc4-40d6-b6f9-c113101efd3f/volumes" Sep 07 17:23:00.428566 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.428553 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8435f5ee-a079-46cd-8a0f-35bb12ef06b3" path="/var/lib/kubelet/pods/8435f5ee-a079-46cd-8a0f-35bb12ef06b3/volumes" Sep 07 17:23:00.760541 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:00.760510 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 17:23:01.053380 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:01.053293 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerID="f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4" exitCode=0 Sep 07 17:23:01.053380 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:01.053341 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerDied","Data":"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4"} Sep 07 17:23:10.760217 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:10.760173 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 17:23:20.760788 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:20.760720 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 17:23:20.761151 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:20.760886 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:23:30.097251 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.097223 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:23:30.161162 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.161129 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls\") pod \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " Sep 07 17:23:30.161319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.161189 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-offline-tls\") pod \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " Sep 07 17:23:30.161385 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.161363 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sdml\" (UniqueName: \"kubernetes.io/projected/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-kube-api-access-6sdml\") pod \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " Sep 07 17:23:30.161428 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.161412 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-online-tls\") pod \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " Sep 07 17:23:30.161474 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.161452 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-feast-data\") pod \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\" (UID: \"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90\") " Sep 07 17:23:30.163298 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.163270 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" (UID: "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:23:30.163444 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.163365 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" (UID: "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:23:30.163444 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.163428 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-kube-api-access-6sdml" (OuterVolumeSpecName: "kube-api-access-6sdml") pod "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" (UID: "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90"). InnerVolumeSpecName "kube-api-access-6sdml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:23:30.163651 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.163634 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-online-tls" (OuterVolumeSpecName: "online-tls") pod "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" (UID: "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:23:30.170746 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.170701 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-feast-data" (OuterVolumeSpecName: "feast-data") pod "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" (UID: "5e10f25e-5903-4fc6-adc3-0eeb4fe17c90"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:23:30.174232 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.174207 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerID="6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce" exitCode=137 Sep 07 17:23:30.174232 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.174232 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerID="f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2" exitCode=137 Sep 07 17:23:30.174378 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.174273 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" Sep 07 17:23:30.174378 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.174270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerDied","Data":"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce"} Sep 07 17:23:30.174378 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.174328 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerDied","Data":"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2"} Sep 07 17:23:30.174378 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.174350 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-776f6db575-kbcw6" event={"ID":"5e10f25e-5903-4fc6-adc3-0eeb4fe17c90","Type":"ContainerDied","Data":"1f1be51c899a42326cfc97b85c7a22d0865a4561688e96ac95d3ebe1c5b70a6c"} Sep 07 17:23:30.174378 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.174371 2563 scope.go:117] "RemoveContainer" containerID="6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce" Sep 07 17:23:30.182765 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.182719 2563 scope.go:117] "RemoveContainer" containerID="f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4" Sep 07 17:23:30.191339 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.191317 2563 scope.go:117] "RemoveContainer" containerID="f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2" Sep 07 17:23:30.196590 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.196567 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-776f6db575-kbcw6"] Sep 07 17:23:30.200329 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.200306 2563 scope.go:117] "RemoveContainer" containerID="273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1" Sep 07 17:23:30.201104 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.201084 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-776f6db575-kbcw6"] Sep 07 17:23:30.207649 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.207631 2563 scope.go:117] "RemoveContainer" containerID="4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a" Sep 07 17:23:30.214444 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.214428 2563 scope.go:117] "RemoveContainer" containerID="6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce" Sep 07 17:23:30.214712 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:23:30.214695 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce\": container with ID starting with 6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce not found: ID does not exist" containerID="6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce" Sep 07 17:23:30.214771 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.214737 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce"} err="failed to get container status \"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce\": rpc error: code = NotFound desc = could not find container \"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce\": container with ID starting with 6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce not found: ID does not exist" Sep 07 17:23:30.214771 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.214757 2563 scope.go:117] "RemoveContainer" containerID="f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4" Sep 07 17:23:30.214990 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:23:30.214973 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4\": container with ID starting with f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4 not found: ID does not exist" containerID="f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4" Sep 07 17:23:30.215059 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215001 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4"} err="failed to get container status \"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4\": rpc error: code = NotFound desc = could not find container \"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4\": container with ID starting with f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4 not found: ID does not exist" Sep 07 17:23:30.215059 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215025 2563 scope.go:117] "RemoveContainer" containerID="f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2" Sep 07 17:23:30.215261 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:23:30.215247 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2\": container with ID starting with f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2 not found: ID does not exist" containerID="f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2" Sep 07 17:23:30.215300 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215265 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2"} err="failed to get container status \"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2\": rpc error: code = NotFound desc = could not find container \"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2\": container with ID starting with f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2 not found: ID does not exist" Sep 07 17:23:30.215300 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215278 2563 scope.go:117] "RemoveContainer" containerID="273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1" Sep 07 17:23:30.215484 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:23:30.215467 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1\": container with ID starting with 273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1 not found: ID does not exist" containerID="273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1" Sep 07 17:23:30.215528 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215489 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1"} err="failed to get container status \"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1\": rpc error: code = NotFound desc = could not find container \"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1\": container with ID starting with 273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1 not found: ID does not exist" Sep 07 17:23:30.215528 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215504 2563 scope.go:117] "RemoveContainer" containerID="4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a" Sep 07 17:23:30.215693 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:23:30.215679 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a\": container with ID starting with 4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a not found: ID does not exist" containerID="4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a" Sep 07 17:23:30.215750 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215695 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a"} err="failed to get container status \"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a\": rpc error: code = NotFound desc = could not find container \"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a\": container with ID starting with 4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a not found: ID does not exist" Sep 07 17:23:30.215750 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215706 2563 scope.go:117] "RemoveContainer" containerID="6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce" Sep 07 17:23:30.215956 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215940 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce"} err="failed to get container status \"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce\": rpc error: code = NotFound desc = could not find container \"6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce\": container with ID starting with 6bf1ae4b579f956b6ca6415375505e781de5dc701aab312dd6d0f6e85ba1c2ce not found: ID does not exist" Sep 07 17:23:30.216002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.215956 2563 scope.go:117] "RemoveContainer" containerID="f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4" Sep 07 17:23:30.216195 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.216178 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4"} err="failed to get container status \"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4\": rpc error: code = NotFound desc = could not find container \"f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4\": container with ID starting with f9e0ff20ba4c5e19c60ee4a0e6ff63493f162f89baf02df4246188a7a78fe6d4 not found: ID does not exist" Sep 07 17:23:30.216238 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.216199 2563 scope.go:117] "RemoveContainer" containerID="f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2" Sep 07 17:23:30.216410 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.216391 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2"} err="failed to get container status \"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2\": rpc error: code = NotFound desc = could not find container \"f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2\": container with ID starting with f633024b15e4ed69fc1b4f001f04ac9db76d3f42b0f9a6bbd9271591dd459cb2 not found: ID does not exist" Sep 07 17:23:30.216454 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.216411 2563 scope.go:117] "RemoveContainer" containerID="273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1" Sep 07 17:23:30.216638 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.216622 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1"} err="failed to get container status \"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1\": rpc error: code = NotFound desc = could not find container \"273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1\": container with ID starting with 273ffb933233b316b1679033d3ac2cf05e3b7f5bfe6fa569fb449c9bf496a0d1 not found: ID does not exist" Sep 07 17:23:30.216638 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.216637 2563 scope.go:117] "RemoveContainer" containerID="4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a" Sep 07 17:23:30.216845 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.216831 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a"} err="failed to get container status \"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a\": rpc error: code = NotFound desc = could not find container \"4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a\": container with ID starting with 4c6d30aa3a60a735dda774a9ee8901818c7e2445f9e20552713703179dd9b03a not found: ID does not exist" Sep 07 17:23:30.262782 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.262749 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sdml\" (UniqueName: \"kubernetes.io/projected/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-kube-api-access-6sdml\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:30.262782 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.262776 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-online-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:30.262782 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.262786 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-feast-data\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:30.263014 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.262793 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-registry-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:30.263014 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.262802 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90-offline-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:23:30.425600 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:30.425567 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" path="/var/lib/kubelet/pods/5e10f25e-5903-4fc6-adc3-0eeb4fe17c90/volumes" Sep 07 17:23:37.089947 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:37.089908 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:23:37.189684 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:37.189641 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:23:37.290316 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:37.290277 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:23:37.348068 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:37.347982 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:23:37.940128 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:23:37.940079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:24:14.352814 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:24:14.352778 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:37.435943 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.435887 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:37.534006 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.533958 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:37.615602 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615570 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-r6hs9"] Sep 07 17:25:37.615894 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615880 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" Sep 07 17:25:37.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615897 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" Sep 07 17:25:37.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615910 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" Sep 07 17:25:37.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615916 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" Sep 07 17:25:37.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615925 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8435f5ee-a079-46cd-8a0f-35bb12ef06b3" containerName="redis" Sep 07 17:25:37.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615930 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8435f5ee-a079-46cd-8a0f-35bb12ef06b3" containerName="redis" Sep 07 17:25:37.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615939 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44631e6c-6cc4-40d6-b6f9-c113101efd3f" containerName="postgres" Sep 07 17:25:37.615946 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615944 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44631e6c-6cc4-40d6-b6f9-c113101efd3f" containerName="postgres" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615952 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="feast-apply" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615957 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="feast-apply" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615965 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="offline" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615970 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="offline" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615976 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="feast-init" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.615982 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="feast-init" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.616025 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="44631e6c-6cc4-40d6-b6f9-c113101efd3f" containerName="postgres" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.616034 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="online" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.616040 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8435f5ee-a079-46cd-8a0f-35bb12ef06b3" containerName="redis" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.616045 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="registry" Sep 07 17:25:37.616171 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.616051 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e10f25e-5903-4fc6-adc3-0eeb4fe17c90" containerName="offline" Sep 07 17:25:37.618960 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.618946 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" Sep 07 17:25:37.621021 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.620997 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 07 17:25:37.621386 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.621366 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 07 17:25:37.621386 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.621378 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-j9dlc\"" Sep 07 17:25:37.643713 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.643684 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:37.664402 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.664377 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-r6hs9"] Sep 07 17:25:37.676182 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.676161 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq7bx\" (UniqueName: \"kubernetes.io/projected/f67998da-9760-4912-99db-46fb3a6f5137-kube-api-access-tq7bx\") pod \"redis-69bbd798b6-r6hs9\" (UID: \"f67998da-9760-4912-99db-46fb3a6f5137\") " pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" Sep 07 17:25:37.776641 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.776612 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq7bx\" (UniqueName: \"kubernetes.io/projected/f67998da-9760-4912-99db-46fb3a6f5137-kube-api-access-tq7bx\") pod \"redis-69bbd798b6-r6hs9\" (UID: \"f67998da-9760-4912-99db-46fb3a6f5137\") " pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" Sep 07 17:25:37.784572 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.784545 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq7bx\" (UniqueName: \"kubernetes.io/projected/f67998da-9760-4912-99db-46fb3a6f5137-kube-api-access-tq7bx\") pod \"redis-69bbd798b6-r6hs9\" (UID: \"f67998da-9760-4912-99db-46fb3a6f5137\") " pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" Sep 07 17:25:37.943241 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:37.943207 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" Sep 07 17:25:38.210348 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:38.210264 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-r6hs9"] Sep 07 17:25:38.214007 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:25:38.213975 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf67998da_9760_4912_99db_46fb3a6f5137.slice/crio-89197e6c618eae6c5f93e13b87f83112555da8cc1e8b2a1a14c027ca4fafbebd WatchSource:0}: Error finding container 89197e6c618eae6c5f93e13b87f83112555da8cc1e8b2a1a14c027ca4fafbebd: Status 404 returned error can't find the container with id 89197e6c618eae6c5f93e13b87f83112555da8cc1e8b2a1a14c027ca4fafbebd Sep 07 17:25:38.644836 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:38.644804 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" event={"ID":"f67998da-9760-4912-99db-46fb3a6f5137","Type":"ContainerStarted","Data":"05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b"} Sep 07 17:25:38.644836 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:38.644844 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" event={"ID":"f67998da-9760-4912-99db-46fb3a6f5137","Type":"ContainerStarted","Data":"89197e6c618eae6c5f93e13b87f83112555da8cc1e8b2a1a14c027ca4fafbebd"} Sep 07 17:25:38.659376 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:38.659324 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" podStartSLOduration=1.659307844 podStartE2EDuration="1.659307844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:25:38.658971419 +0000 UTC m=+1046.731872762" watchObservedRunningTime="2026-09-07 17:25:38.659307844 +0000 UTC m=+1046.732209178" Sep 07 17:25:43.051906 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.051864 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-knpxr"] Sep 07 17:25:43.055555 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.055536 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.057351 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.057329 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 07 17:25:43.060550 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.060528 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-knpxr"] Sep 07 17:25:43.087123 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.087092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:43.121207 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.121181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1f968a9f-3ff8-4a31-92f5-7a303c889225-postgresdata\") pod \"postgres-864846d4f5-knpxr\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.121412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.121308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pvn8\" (UniqueName: \"kubernetes.io/projected/1f968a9f-3ff8-4a31-92f5-7a303c889225-kube-api-access-7pvn8\") pod \"postgres-864846d4f5-knpxr\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.222141 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.222107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pvn8\" (UniqueName: \"kubernetes.io/projected/1f968a9f-3ff8-4a31-92f5-7a303c889225-kube-api-access-7pvn8\") pod \"postgres-864846d4f5-knpxr\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.222303 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.222149 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1f968a9f-3ff8-4a31-92f5-7a303c889225-postgresdata\") pod \"postgres-864846d4f5-knpxr\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.222498 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.222481 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1f968a9f-3ff8-4a31-92f5-7a303c889225-postgresdata\") pod \"postgres-864846d4f5-knpxr\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.230034 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.230010 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pvn8\" (UniqueName: \"kubernetes.io/projected/1f968a9f-3ff8-4a31-92f5-7a303c889225-kube-api-access-7pvn8\") pod \"postgres-864846d4f5-knpxr\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.365549 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.365469 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:25:43.632273 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.632193 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-knpxr"] Sep 07 17:25:43.635877 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:25:43.635851 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f968a9f_3ff8_4a31_92f5_7a303c889225.slice/crio-02b5804fbf3e85abe937ba1172ac8895a51a2f536ade48dfee589b3a7785f262 WatchSource:0}: Error finding container 02b5804fbf3e85abe937ba1172ac8895a51a2f536ade48dfee589b3a7785f262: Status 404 returned error can't find the container with id 02b5804fbf3e85abe937ba1172ac8895a51a2f536ade48dfee589b3a7785f262 Sep 07 17:25:43.663412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:43.663381 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" event={"ID":"1f968a9f-3ff8-4a31-92f5-7a303c889225","Type":"ContainerStarted","Data":"02b5804fbf3e85abe937ba1172ac8895a51a2f536ade48dfee589b3a7785f262"} Sep 07 17:25:44.669036 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:44.668997 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" event={"ID":"1f968a9f-3ff8-4a31-92f5-7a303c889225","Type":"ContainerStarted","Data":"bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104"} Sep 07 17:25:44.684715 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:44.684668 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" podStartSLOduration=1.684652614 podStartE2EDuration="1.684652614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:25:44.68414128 +0000 UTC m=+1052.757042615" watchObservedRunningTime="2026-09-07 17:25:44.684652614 +0000 UTC m=+1052.757553947" Sep 07 17:25:48.550052 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.550016 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:48.636262 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.636222 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:48.636496 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.636479 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2"] Sep 07 17:25:48.640176 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.640152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.641987 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.641963 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 07 17:25:48.642127 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.642058 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 07 17:25:48.642127 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.642012 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 07 17:25:48.642127 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.642027 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-d6rhz\"" Sep 07 17:25:48.651685 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.651662 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2"] Sep 07 17:25:48.770915 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.770874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-online-tls\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.771080 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.770934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-registry-tls\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.771080 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.770972 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc85ce8-a665-466e-87ff-cdb2e68013c0-feast-data\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.771080 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.770998 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57sr6\" (UniqueName: \"kubernetes.io/projected/4bc85ce8-a665-466e-87ff-cdb2e68013c0-kube-api-access-57sr6\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.779055 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.779027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:48.872481 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.872399 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-online-tls\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.872481 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.872441 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-registry-tls\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.872704 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.872556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc85ce8-a665-466e-87ff-cdb2e68013c0-feast-data\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.872704 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.872595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57sr6\" (UniqueName: \"kubernetes.io/projected/4bc85ce8-a665-466e-87ff-cdb2e68013c0-kube-api-access-57sr6\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.872982 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.872961 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc85ce8-a665-466e-87ff-cdb2e68013c0-feast-data\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.874870 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.874846 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-online-tls\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.874988 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.874918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-registry-tls\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.879467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.879438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:25:48.880415 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.880395 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57sr6\" (UniqueName: \"kubernetes.io/projected/4bc85ce8-a665-466e-87ff-cdb2e68013c0-kube-api-access-57sr6\") pod \"feast-credit-scoring-69dc88fd85-wdsz2\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:48.950093 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:48.950050 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:25:49.226407 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:49.226330 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2"] Sep 07 17:25:49.230885 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:25:49.230861 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc85ce8_a665_466e_87ff_cdb2e68013c0.slice/crio-71c93b6abce8653cadc5fe5ea43b17bc575b4eec2ac1d5d4d89e5f473d80a60c WatchSource:0}: Error finding container 71c93b6abce8653cadc5fe5ea43b17bc575b4eec2ac1d5d4d89e5f473d80a60c: Status 404 returned error can't find the container with id 71c93b6abce8653cadc5fe5ea43b17bc575b4eec2ac1d5d4d89e5f473d80a60c Sep 07 17:25:49.689130 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:49.689097 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerStarted","Data":"221bfd6cb1311d43f9977363e00d0c5eb02b4951721e96f00f5d796bc3c59f2c"} Sep 07 17:25:49.689507 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:49.689145 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerStarted","Data":"71c93b6abce8653cadc5fe5ea43b17bc575b4eec2ac1d5d4d89e5f473d80a60c"} Sep 07 17:25:51.697746 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:51.697693 2563 generic.go:356] "Generic (PLEG): container finished" podID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerID="221bfd6cb1311d43f9977363e00d0c5eb02b4951721e96f00f5d796bc3c59f2c" exitCode=0 Sep 07 17:25:51.698121 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:51.697777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerDied","Data":"221bfd6cb1311d43f9977363e00d0c5eb02b4951721e96f00f5d796bc3c59f2c"} Sep 07 17:25:52.704198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:52.704151 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerStarted","Data":"396bf908028774757fdf2934ec1740e4735cf1c63d800c243fe920c518662374"} Sep 07 17:25:58.729364 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:58.729328 2563 generic.go:356] "Generic (PLEG): container finished" podID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerID="396bf908028774757fdf2934ec1740e4735cf1c63d800c243fe920c518662374" exitCode=0 Sep 07 17:25:58.729681 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:58.729400 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerDied","Data":"396bf908028774757fdf2934ec1740e4735cf1c63d800c243fe920c518662374"} Sep 07 17:25:59.737188 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:59.737146 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerStarted","Data":"99aba7cbd5ebe05de7108aee5460e65f884b54f45900fc870e0d7fb9eb03e880"} Sep 07 17:25:59.737718 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:59.737224 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerStarted","Data":"29174b8dbeaa7f7ec526757b39491d0463e97caa7796ee6f02ac4bb97bda5aa4"} Sep 07 17:25:59.758222 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:25:59.758158 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podStartSLOduration=11.758138069 podStartE2EDuration="11.758138069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:25:59.757625005 +0000 UTC m=+1067.830526343" watchObservedRunningTime="2026-09-07 17:25:59.758138069 +0000 UTC m=+1067.831039403" Sep 07 17:26:00.950810 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:00.950770 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:00.951262 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:00.950842 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:00.952585 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:00.952550 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 07 17:26:00.952585 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:00.952579 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 07 17:26:03.950467 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:03.950415 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 07 17:26:03.951010 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:03.950429 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 07 17:26:06.950717 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:06.950619 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 07 17:26:06.956861 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:06.956828 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:06.957412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:06.957384 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:06.960279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:06.960259 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:08.950700 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:08.950664 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:09.950363 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:09.950321 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 07 17:26:12.951842 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:12.951810 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:12.952572 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:12.952550 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:26:13.264342 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:13.264287 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:26:14.427568 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.427531 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:26:14.520245 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.520207 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:26:14.524191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.524167 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq"] Sep 07 17:26:14.528639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.528620 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.531502 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.531478 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 07 17:26:14.531637 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.531610 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 07 17:26:14.531740 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.531714 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 07 17:26:14.532191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.532175 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-9wk5h\"" Sep 07 17:26:14.541665 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.541646 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq"] Sep 07 17:26:14.592718 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.592695 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.592847 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.592736 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44jsr\" (UniqueName: \"kubernetes.io/projected/ed67eb91-db64-4602-b08e-3bd01b2a379a-kube-api-access-44jsr\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.592847 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.592755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed67eb91-db64-4602-b08e-3bd01b2a379a-feast-data\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.592921 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.592848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-registry-tls\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.628482 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.628455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:26:14.693441 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.693362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.693441 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.693401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44jsr\" (UniqueName: \"kubernetes.io/projected/ed67eb91-db64-4602-b08e-3bd01b2a379a-kube-api-access-44jsr\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.693441 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.693423 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed67eb91-db64-4602-b08e-3bd01b2a379a-feast-data\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.693659 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.693460 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-registry-tls\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.693659 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:26:14.693635 2563 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 07 17:26:14.693764 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:26:14.693709 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls podName:ed67eb91-db64-4602-b08e-3bd01b2a379a nodeName:}" failed. No retries permitted until 2026-09-07 17:26:15.193685972 +0000 UTC m=+1083.266587303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls") pod "feast-driver-ranking-5f6c998ccd-2l7kq" (UID: "ed67eb91-db64-4602-b08e-3bd01b2a379a") : secret "feast-driver-ranking-online-tls" not found Sep 07 17:26:14.693876 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.693854 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed67eb91-db64-4602-b08e-3bd01b2a379a-feast-data\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.695893 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.695875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-registry-tls\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.703820 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.703796 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44jsr\" (UniqueName: \"kubernetes.io/projected/ed67eb91-db64-4602-b08e-3bd01b2a379a-kube-api-access-44jsr\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:14.842195 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:14.842151 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:26:15.197684 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:15.197644 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:15.200037 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:15.200009 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls\") pod \"feast-driver-ranking-5f6c998ccd-2l7kq\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:15.438310 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:15.438271 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:15.715332 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:15.715304 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq"] Sep 07 17:26:15.717011 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:26:15.716983 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded67eb91_db64_4602_b08e_3bd01b2a379a.slice/crio-2ffdc306a44416dbcc0fd9b5cf444a0bccf091aa6146554a5b28e1329c339932 WatchSource:0}: Error finding container 2ffdc306a44416dbcc0fd9b5cf444a0bccf091aa6146554a5b28e1329c339932: Status 404 returned error can't find the container with id 2ffdc306a44416dbcc0fd9b5cf444a0bccf091aa6146554a5b28e1329c339932 Sep 07 17:26:15.810518 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:15.810487 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerStarted","Data":"b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228"} Sep 07 17:26:15.810657 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:15.810538 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerStarted","Data":"2ffdc306a44416dbcc0fd9b5cf444a0bccf091aa6146554a5b28e1329c339932"} Sep 07 17:26:20.832773 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:20.832672 2563 generic.go:356] "Generic (PLEG): container finished" podID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerID="b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228" exitCode=0 Sep 07 17:26:20.832773 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:20.832713 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerDied","Data":"b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228"} Sep 07 17:26:21.838976 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:21.838942 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerStarted","Data":"5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd"} Sep 07 17:26:27.866766 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:27.866718 2563 generic.go:356] "Generic (PLEG): container finished" podID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerID="5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd" exitCode=0 Sep 07 17:26:27.867257 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:27.866795 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerDied","Data":"5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd"} Sep 07 17:26:28.874416 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:28.874381 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerStarted","Data":"30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4"} Sep 07 17:26:28.874799 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:28.874444 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerStarted","Data":"343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86"} Sep 07 17:26:28.895548 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:28.895501 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podStartSLOduration=14.8954828 podStartE2EDuration="14.8954828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:26:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:26:28.894671744 +0000 UTC m=+1096.967573076" watchObservedRunningTime="2026-09-07 17:26:28.8954828 +0000 UTC m=+1096.968384138" Sep 07 17:26:30.439393 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:30.439361 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:30.439881 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:30.439423 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:30.440963 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:30.440930 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 07 17:26:30.441106 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:30.440977 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 07 17:26:33.438610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:33.438554 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 07 17:26:33.438610 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:33.438584 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 07 17:26:35.438854 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:35.438808 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:35.439221 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:35.438885 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:36.438932 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:36.438889 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 07 17:26:36.447266 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:36.447241 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:36.451780 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:36.451760 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:39.439582 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:39.439531 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 07 17:26:42.438926 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:42.438895 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:26:42.439645 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:26:42.439626 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:28:00.683914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:00.683875 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:02.653914 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:02.653880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:03.046402 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.046372 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:03.050341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.050316 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj"] Sep 07 17:28:03.050544 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.050519 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" podUID="0696698b-9f70-4540-aa3a-e85f49c513c3" containerName="manager" containerID="cri-o://e850e2692bf7ee9d1a25a53cf3dfc044ac28b7a22f3e04a9e50892ab4b52b58e" gracePeriod=10 Sep 07 17:28:03.279511 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.279461 2563 generic.go:356] "Generic (PLEG): container finished" podID="0696698b-9f70-4540-aa3a-e85f49c513c3" containerID="e850e2692bf7ee9d1a25a53cf3dfc044ac28b7a22f3e04a9e50892ab4b52b58e" exitCode=0 Sep 07 17:28:03.279636 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.279505 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" event={"ID":"0696698b-9f70-4540-aa3a-e85f49c513c3","Type":"ContainerDied","Data":"e850e2692bf7ee9d1a25a53cf3dfc044ac28b7a22f3e04a9e50892ab4b52b58e"} Sep 07 17:28:03.297906 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.297840 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:28:03.312867 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.312839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:03.351474 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.351443 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2"] Sep 07 17:28:03.351752 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.351709 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" containerID="cri-o://29174b8dbeaa7f7ec526757b39491d0463e97caa7796ee6f02ac4bb97bda5aa4" gracePeriod=30 Sep 07 17:28:03.351879 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.351785 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" containerID="cri-o://99aba7cbd5ebe05de7108aee5460e65f884b54f45900fc870e0d7fb9eb03e880" gracePeriod=30 Sep 07 17:28:03.364038 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.364007 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq"] Sep 07 17:28:03.364297 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.364268 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" containerID="cri-o://343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86" gracePeriod=30 Sep 07 17:28:03.364388 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.364317 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" containerID="cri-o://30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4" gracePeriod=30 Sep 07 17:28:03.384949 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.384924 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:03.394108 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.394083 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr8ph\" (UniqueName: \"kubernetes.io/projected/0696698b-9f70-4540-aa3a-e85f49c513c3-kube-api-access-jr8ph\") pod \"0696698b-9f70-4540-aa3a-e85f49c513c3\" (UID: \"0696698b-9f70-4540-aa3a-e85f49c513c3\") " Sep 07 17:28:03.396076 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.396055 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0696698b-9f70-4540-aa3a-e85f49c513c3-kube-api-access-jr8ph" (OuterVolumeSpecName: "kube-api-access-jr8ph") pod "0696698b-9f70-4540-aa3a-e85f49c513c3" (UID: "0696698b-9f70-4540-aa3a-e85f49c513c3"). InnerVolumeSpecName "kube-api-access-jr8ph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:28:03.494920 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:03.494876 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr8ph\" (UniqueName: \"kubernetes.io/projected/0696698b-9f70-4540-aa3a-e85f49c513c3-kube-api-access-jr8ph\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:04.285299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.285274 2563 generic.go:356] "Generic (PLEG): container finished" podID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerID="99aba7cbd5ebe05de7108aee5460e65f884b54f45900fc870e0d7fb9eb03e880" exitCode=0 Sep 07 17:28:04.285651 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.285332 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerDied","Data":"99aba7cbd5ebe05de7108aee5460e65f884b54f45900fc870e0d7fb9eb03e880"} Sep 07 17:28:04.286518 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.286501 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" Sep 07 17:28:04.286619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.286499 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj" event={"ID":"0696698b-9f70-4540-aa3a-e85f49c513c3","Type":"ContainerDied","Data":"5036a3ab8902b40a916b6aa95374406d9a03ad5191a4658673aaae3e8ab2a986"} Sep 07 17:28:04.286674 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.286617 2563 scope.go:117] "RemoveContainer" containerID="e850e2692bf7ee9d1a25a53cf3dfc044ac28b7a22f3e04a9e50892ab4b52b58e" Sep 07 17:28:04.289285 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.289260 2563 generic.go:356] "Generic (PLEG): container finished" podID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerID="30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4" exitCode=0 Sep 07 17:28:04.289382 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.289327 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerDied","Data":"30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4"} Sep 07 17:28:04.307801 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.307778 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj"] Sep 07 17:28:04.315360 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.315338 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-695d6b979d-p8gkj"] Sep 07 17:28:04.425617 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:04.425526 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0696698b-9f70-4540-aa3a-e85f49c513c3" path="/var/lib/kubelet/pods/0696698b-9f70-4540-aa3a-e85f49c513c3/volumes" Sep 07 17:28:05.627022 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.626985 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-knpxr"] Sep 07 17:28:05.627405 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.627148 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" podUID="1f968a9f-3ff8-4a31-92f5-7a303c889225" containerName="postgres" containerID="cri-o://bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104" gracePeriod=30 Sep 07 17:28:05.638503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.638471 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-r6hs9"] Sep 07 17:28:05.638689 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.638663 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" podUID="f67998da-9760-4912-99db-46fb3a6f5137" containerName="redis" containerID="cri-o://05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b" gracePeriod=30 Sep 07 17:28:05.701533 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.701502 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:05.894289 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.894265 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:28:05.896845 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.896826 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" Sep 07 17:28:05.915395 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.915365 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pvn8\" (UniqueName: \"kubernetes.io/projected/1f968a9f-3ff8-4a31-92f5-7a303c889225-kube-api-access-7pvn8\") pod \"1f968a9f-3ff8-4a31-92f5-7a303c889225\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " Sep 07 17:28:05.915531 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.915453 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1f968a9f-3ff8-4a31-92f5-7a303c889225-postgresdata\") pod \"1f968a9f-3ff8-4a31-92f5-7a303c889225\" (UID: \"1f968a9f-3ff8-4a31-92f5-7a303c889225\") " Sep 07 17:28:05.915531 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.915486 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq7bx\" (UniqueName: \"kubernetes.io/projected/f67998da-9760-4912-99db-46fb3a6f5137-kube-api-access-tq7bx\") pod \"f67998da-9760-4912-99db-46fb3a6f5137\" (UID: \"f67998da-9760-4912-99db-46fb3a6f5137\") " Sep 07 17:28:05.918007 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.917983 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f968a9f-3ff8-4a31-92f5-7a303c889225-kube-api-access-7pvn8" (OuterVolumeSpecName: "kube-api-access-7pvn8") pod "1f968a9f-3ff8-4a31-92f5-7a303c889225" (UID: "1f968a9f-3ff8-4a31-92f5-7a303c889225"). InnerVolumeSpecName "kube-api-access-7pvn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:28:05.918125 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.918114 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f67998da-9760-4912-99db-46fb3a6f5137-kube-api-access-tq7bx" (OuterVolumeSpecName: "kube-api-access-tq7bx") pod "f67998da-9760-4912-99db-46fb3a6f5137" (UID: "f67998da-9760-4912-99db-46fb3a6f5137"). InnerVolumeSpecName "kube-api-access-tq7bx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:28:05.948133 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:05.948094 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f968a9f-3ff8-4a31-92f5-7a303c889225-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "1f968a9f-3ff8-4a31-92f5-7a303c889225" (UID: "1f968a9f-3ff8-4a31-92f5-7a303c889225"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:28:06.016101 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.016067 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pvn8\" (UniqueName: \"kubernetes.io/projected/1f968a9f-3ff8-4a31-92f5-7a303c889225-kube-api-access-7pvn8\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:06.016101 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.016096 2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1f968a9f-3ff8-4a31-92f5-7a303c889225-postgresdata\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:06.016101 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.016107 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq7bx\" (UniqueName: \"kubernetes.io/projected/f67998da-9760-4912-99db-46fb3a6f5137-kube-api-access-tq7bx\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:06.298742 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.298688 2563 generic.go:356] "Generic (PLEG): container finished" podID="1f968a9f-3ff8-4a31-92f5-7a303c889225" containerID="bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104" exitCode=0 Sep 07 17:28:06.298930 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.298758 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" Sep 07 17:28:06.298930 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.298777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" event={"ID":"1f968a9f-3ff8-4a31-92f5-7a303c889225","Type":"ContainerDied","Data":"bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104"} Sep 07 17:28:06.298930 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.298819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-knpxr" event={"ID":"1f968a9f-3ff8-4a31-92f5-7a303c889225","Type":"ContainerDied","Data":"02b5804fbf3e85abe937ba1172ac8895a51a2f536ade48dfee589b3a7785f262"} Sep 07 17:28:06.298930 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.298841 2563 scope.go:117] "RemoveContainer" containerID="bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104" Sep 07 17:28:06.300075 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.300050 2563 generic.go:356] "Generic (PLEG): container finished" podID="f67998da-9760-4912-99db-46fb3a6f5137" containerID="05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b" exitCode=0 Sep 07 17:28:06.300156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.300085 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" Sep 07 17:28:06.300156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.300090 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" event={"ID":"f67998da-9760-4912-99db-46fb3a6f5137","Type":"ContainerDied","Data":"05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b"} Sep 07 17:28:06.300156 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.300120 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-r6hs9" event={"ID":"f67998da-9760-4912-99db-46fb3a6f5137","Type":"ContainerDied","Data":"89197e6c618eae6c5f93e13b87f83112555da8cc1e8b2a1a14c027ca4fafbebd"} Sep 07 17:28:06.307604 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.307577 2563 scope.go:117] "RemoveContainer" containerID="bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104" Sep 07 17:28:06.307917 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:28:06.307897 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104\": container with ID starting with bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104 not found: ID does not exist" containerID="bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104" Sep 07 17:28:06.307991 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.307925 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104"} err="failed to get container status \"bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104\": rpc error: code = NotFound desc = could not find container \"bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104\": container with ID starting with bf6bb2ba3f179cbc8e323c59c3a1b23e1f9da6062832ba31386157bb3d0c5104 not found: ID does not exist" Sep 07 17:28:06.307991 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.307943 2563 scope.go:117] "RemoveContainer" containerID="05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b" Sep 07 17:28:06.315426 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.315409 2563 scope.go:117] "RemoveContainer" containerID="05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b" Sep 07 17:28:06.315708 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:28:06.315691 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b\": container with ID starting with 05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b not found: ID does not exist" containerID="05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b" Sep 07 17:28:06.315830 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.315716 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b"} err="failed to get container status \"05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b\": rpc error: code = NotFound desc = could not find container \"05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b\": container with ID starting with 05bcd9488ce1a83c4ab05a7b6e5cf79598b29ec4c7a4547d11dae24d07fa633b not found: ID does not exist" Sep 07 17:28:06.322306 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.322281 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-r6hs9"] Sep 07 17:28:06.325666 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.325645 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-r6hs9"] Sep 07 17:28:06.335542 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.335520 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-knpxr"] Sep 07 17:28:06.341782 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.341758 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-knpxr"] Sep 07 17:28:06.425856 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.425823 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f968a9f-3ff8-4a31-92f5-7a303c889225" path="/var/lib/kubelet/pods/1f968a9f-3ff8-4a31-92f5-7a303c889225/volumes" Sep 07 17:28:06.426147 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.426133 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f67998da-9760-4912-99db-46fb3a6f5137" path="/var/lib/kubelet/pods/f67998da-9760-4912-99db-46fb3a6f5137/volumes" Sep 07 17:28:06.448366 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.448333 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 07 17:28:06.957996 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:06.957899 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 07 17:28:08.612002 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:08.611967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:08.676389 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:08.676360 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:14.080755 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.080672 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:14.173588 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.173548 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:14.618687 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.618652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:14.629448 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629412 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd"] Sep 07 17:28:14.629718 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629707 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f67998da-9760-4912-99db-46fb3a6f5137" containerName="redis" Sep 07 17:28:14.629800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629720 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f67998da-9760-4912-99db-46fb3a6f5137" containerName="redis" Sep 07 17:28:14.629800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629750 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0696698b-9f70-4540-aa3a-e85f49c513c3" containerName="manager" Sep 07 17:28:14.629800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629755 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0696698b-9f70-4540-aa3a-e85f49c513c3" containerName="manager" Sep 07 17:28:14.629800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629762 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f968a9f-3ff8-4a31-92f5-7a303c889225" containerName="postgres" Sep 07 17:28:14.629800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629768 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f968a9f-3ff8-4a31-92f5-7a303c889225" containerName="postgres" Sep 07 17:28:14.629955 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629812 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f968a9f-3ff8-4a31-92f5-7a303c889225" containerName="postgres" Sep 07 17:28:14.629955 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629821 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0696698b-9f70-4540-aa3a-e85f49c513c3" containerName="manager" Sep 07 17:28:14.629955 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.629829 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f67998da-9760-4912-99db-46fb3a6f5137" containerName="redis" Sep 07 17:28:14.634104 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.634086 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:28:14.636049 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.636029 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 07 17:28:14.636341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.636322 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 07 17:28:14.636446 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.636344 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-ntq8g\"" Sep 07 17:28:14.642860 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.642838 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd"] Sep 07 17:28:14.688511 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.688485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psp95\" (UniqueName: \"kubernetes.io/projected/5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f-kube-api-access-psp95\") pod \"feast-operator-controller-manager-689b877b4b-5xxjd\" (UID: \"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:28:14.789229 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.789191 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psp95\" (UniqueName: \"kubernetes.io/projected/5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f-kube-api-access-psp95\") pod \"feast-operator-controller-manager-689b877b4b-5xxjd\" (UID: \"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:28:14.799017 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.798982 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psp95\" (UniqueName: \"kubernetes.io/projected/5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f-kube-api-access-psp95\") pod \"feast-operator-controller-manager-689b877b4b-5xxjd\" (UID: \"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:28:14.945132 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:14.945033 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:28:15.221123 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:15.221009 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd"] Sep 07 17:28:15.224825 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:28:15.224794 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5912794e_98f1_4fb4_9b40_f4aa9e0f0f2f.slice/crio-4c91005995d3f1537cb8073ad1ef8b7459a9728ce72f8495290b6f6ba3e2bc33 WatchSource:0}: Error finding container 4c91005995d3f1537cb8073ad1ef8b7459a9728ce72f8495290b6f6ba3e2bc33: Status 404 returned error can't find the container with id 4c91005995d3f1537cb8073ad1ef8b7459a9728ce72f8495290b6f6ba3e2bc33 Sep 07 17:28:15.226672 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:15.226650 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 17:28:15.336217 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:15.336183 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" event={"ID":"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f","Type":"ContainerStarted","Data":"4c91005995d3f1537cb8073ad1ef8b7459a9728ce72f8495290b6f6ba3e2bc33"} Sep 07 17:28:16.448403 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:16.448364 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 07 17:28:16.958103 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:16.958056 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 07 17:28:18.351627 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:18.351587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" event={"ID":"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f","Type":"ContainerStarted","Data":"cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e"} Sep 07 17:28:18.352065 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:18.351655 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:28:18.377385 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:18.377338 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" podStartSLOduration=2.079411024 podStartE2EDuration="4.377323227s" totalImagesPullingTime="2.297912203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:28:18.376271204 +0000 UTC m=+1206.449172538" watchObservedRunningTime="2026-09-07 17:28:18.377323227 +0000 UTC m=+1206.450224559" Sep 07 17:28:22.072979 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:22.072946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:26.447906 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:26.447864 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 07 17:28:26.448289 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:26.447979 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:28:26.958374 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:26.958333 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 07 17:28:26.958525 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:26.958450 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:28:29.358473 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:29.358437 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:28:30.864760 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:30.864699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:30.947884 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:30.947843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:31.424630 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:31.424596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:31.525371 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:31.525339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:31.679717 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:31.679631 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:31.840967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:31.840938 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:31.998119 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:31.998086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:32.133246 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:32.133212 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:33.410629 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:33.410601 2563 generic.go:356] "Generic (PLEG): container finished" podID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerID="29174b8dbeaa7f7ec526757b39491d0463e97caa7796ee6f02ac4bb97bda5aa4" exitCode=137 Sep 07 17:28:33.410957 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:33.410675 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerDied","Data":"29174b8dbeaa7f7ec526757b39491d0463e97caa7796ee6f02ac4bb97bda5aa4"} Sep 07 17:28:33.999375 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:33.999345 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:28:34.028917 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.028893 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:28:34.158172 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158073 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls\") pod \"ed67eb91-db64-4602-b08e-3bd01b2a379a\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " Sep 07 17:28:34.158172 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158127 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc85ce8-a665-466e-87ff-cdb2e68013c0-feast-data\") pod \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " Sep 07 17:28:34.158172 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158146 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57sr6\" (UniqueName: \"kubernetes.io/projected/4bc85ce8-a665-466e-87ff-cdb2e68013c0-kube-api-access-57sr6\") pod \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " Sep 07 17:28:34.158172 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158171 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-online-tls\") pod \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " Sep 07 17:28:34.158503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158219 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-registry-tls\") pod \"ed67eb91-db64-4602-b08e-3bd01b2a379a\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " Sep 07 17:28:34.158503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158252 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed67eb91-db64-4602-b08e-3bd01b2a379a-feast-data\") pod \"ed67eb91-db64-4602-b08e-3bd01b2a379a\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " Sep 07 17:28:34.158503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158290 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44jsr\" (UniqueName: \"kubernetes.io/projected/ed67eb91-db64-4602-b08e-3bd01b2a379a-kube-api-access-44jsr\") pod \"ed67eb91-db64-4602-b08e-3bd01b2a379a\" (UID: \"ed67eb91-db64-4602-b08e-3bd01b2a379a\") " Sep 07 17:28:34.158503 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158346 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-registry-tls\") pod \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\" (UID: \"4bc85ce8-a665-466e-87ff-cdb2e68013c0\") " Sep 07 17:28:34.158909 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.158839 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed67eb91-db64-4602-b08e-3bd01b2a379a-feast-data" (OuterVolumeSpecName: "feast-data") pod "ed67eb91-db64-4602-b08e-3bd01b2a379a" (UID: "ed67eb91-db64-4602-b08e-3bd01b2a379a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:28:34.160433 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.160399 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ed67eb91-db64-4602-b08e-3bd01b2a379a" (UID: "ed67eb91-db64-4602-b08e-3bd01b2a379a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:28:34.160678 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.160641 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-online-tls" (OuterVolumeSpecName: "online-tls") pod "4bc85ce8-a665-466e-87ff-cdb2e68013c0" (UID: "4bc85ce8-a665-466e-87ff-cdb2e68013c0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:28:34.160786 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.160723 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls" (OuterVolumeSpecName: "online-tls") pod "ed67eb91-db64-4602-b08e-3bd01b2a379a" (UID: "ed67eb91-db64-4602-b08e-3bd01b2a379a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:28:34.160947 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.160900 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4bc85ce8-a665-466e-87ff-cdb2e68013c0" (UID: "4bc85ce8-a665-466e-87ff-cdb2e68013c0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 17:28:34.161032 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.160995 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed67eb91-db64-4602-b08e-3bd01b2a379a-kube-api-access-44jsr" (OuterVolumeSpecName: "kube-api-access-44jsr") pod "ed67eb91-db64-4602-b08e-3bd01b2a379a" (UID: "ed67eb91-db64-4602-b08e-3bd01b2a379a"). InnerVolumeSpecName "kube-api-access-44jsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:28:34.161032 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.161026 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc85ce8-a665-466e-87ff-cdb2e68013c0-kube-api-access-57sr6" (OuterVolumeSpecName: "kube-api-access-57sr6") pod "4bc85ce8-a665-466e-87ff-cdb2e68013c0" (UID: "4bc85ce8-a665-466e-87ff-cdb2e68013c0"). InnerVolumeSpecName "kube-api-access-57sr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:28:34.167397 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.167373 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc85ce8-a665-466e-87ff-cdb2e68013c0-feast-data" (OuterVolumeSpecName: "feast-data") pod "4bc85ce8-a665-466e-87ff-cdb2e68013c0" (UID: "4bc85ce8-a665-466e-87ff-cdb2e68013c0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:28:34.258939 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258908 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-registry-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.258939 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258932 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed67eb91-db64-4602-b08e-3bd01b2a379a-feast-data\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.258939 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258941 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44jsr\" (UniqueName: \"kubernetes.io/projected/ed67eb91-db64-4602-b08e-3bd01b2a379a-kube-api-access-44jsr\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.259167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258951 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-registry-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.259167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258959 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed67eb91-db64-4602-b08e-3bd01b2a379a-online-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.259167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258966 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc85ce8-a665-466e-87ff-cdb2e68013c0-feast-data\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.259167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258974 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57sr6\" (UniqueName: \"kubernetes.io/projected/4bc85ce8-a665-466e-87ff-cdb2e68013c0-kube-api-access-57sr6\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.259167 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.258982 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4bc85ce8-a665-466e-87ff-cdb2e68013c0-online-tls\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:28:34.416480 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.416394 2563 generic.go:356] "Generic (PLEG): container finished" podID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerID="343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86" exitCode=137 Sep 07 17:28:34.416480 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.416469 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" Sep 07 17:28:34.416967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.416477 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerDied","Data":"343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86"} Sep 07 17:28:34.416967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.416563 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq" event={"ID":"ed67eb91-db64-4602-b08e-3bd01b2a379a","Type":"ContainerDied","Data":"2ffdc306a44416dbcc0fd9b5cf444a0bccf091aa6146554a5b28e1329c339932"} Sep 07 17:28:34.416967 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.416596 2563 scope.go:117] "RemoveContainer" containerID="30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4" Sep 07 17:28:34.418960 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.418935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" event={"ID":"4bc85ce8-a665-466e-87ff-cdb2e68013c0","Type":"ContainerDied","Data":"71c93b6abce8653cadc5fe5ea43b17bc575b4eec2ac1d5d4d89e5f473d80a60c"} Sep 07 17:28:34.419047 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.418981 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2" Sep 07 17:28:34.426440 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.426420 2563 scope.go:117] "RemoveContainer" containerID="343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86" Sep 07 17:28:34.435237 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.435221 2563 scope.go:117] "RemoveContainer" containerID="5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd" Sep 07 17:28:34.442966 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.442935 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq"] Sep 07 17:28:34.443025 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.442972 2563 scope.go:117] "RemoveContainer" containerID="b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228" Sep 07 17:28:34.447333 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.447311 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-5f6c998ccd-2l7kq"] Sep 07 17:28:34.450518 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.450500 2563 scope.go:117] "RemoveContainer" containerID="30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4" Sep 07 17:28:34.450803 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:28:34.450781 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4\": container with ID starting with 30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4 not found: ID does not exist" containerID="30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4" Sep 07 17:28:34.450896 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.450812 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4"} err="failed to get container status \"30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4\": rpc error: code = NotFound desc = could not find container \"30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4\": container with ID starting with 30e1a3f57335b5b962b960c3fc55d206678fb501538bcb987ea2b50506717be4 not found: ID does not exist" Sep 07 17:28:34.450896 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.450835 2563 scope.go:117] "RemoveContainer" containerID="343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86" Sep 07 17:28:34.451147 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:28:34.451129 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86\": container with ID starting with 343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86 not found: ID does not exist" containerID="343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86" Sep 07 17:28:34.451188 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.451152 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86"} err="failed to get container status \"343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86\": rpc error: code = NotFound desc = could not find container \"343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86\": container with ID starting with 343184f4e1c5f8cc7629628395121bb8e56ed4016470b6d2ba6795fbf61f3a86 not found: ID does not exist" Sep 07 17:28:34.451188 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.451168 2563 scope.go:117] "RemoveContainer" containerID="5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd" Sep 07 17:28:34.451419 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:28:34.451401 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd\": container with ID starting with 5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd not found: ID does not exist" containerID="5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd" Sep 07 17:28:34.451488 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.451428 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd"} err="failed to get container status \"5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd\": rpc error: code = NotFound desc = could not find container \"5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd\": container with ID starting with 5b87ef0115226960841e80ea0dfae9aecd0b13b9510164e6448d04582e6aacbd not found: ID does not exist" Sep 07 17:28:34.451488 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.451448 2563 scope.go:117] "RemoveContainer" containerID="b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228" Sep 07 17:28:34.451710 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:28:34.451693 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228\": container with ID starting with b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228 not found: ID does not exist" containerID="b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228" Sep 07 17:28:34.451766 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.451716 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228"} err="failed to get container status \"b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228\": rpc error: code = NotFound desc = could not find container \"b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228\": container with ID starting with b901a0495d3f53019ed3059b74dac28fc3a3152084471c7f39c97f51906a4228 not found: ID does not exist" Sep 07 17:28:34.451766 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.451747 2563 scope.go:117] "RemoveContainer" containerID="99aba7cbd5ebe05de7108aee5460e65f884b54f45900fc870e0d7fb9eb03e880" Sep 07 17:28:34.456947 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.456927 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2"] Sep 07 17:28:34.459840 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.459787 2563 scope.go:117] "RemoveContainer" containerID="29174b8dbeaa7f7ec526757b39491d0463e97caa7796ee6f02ac4bb97bda5aa4" Sep 07 17:28:34.461461 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.461440 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-69dc88fd85-wdsz2"] Sep 07 17:28:34.467639 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.467617 2563 scope.go:117] "RemoveContainer" containerID="396bf908028774757fdf2934ec1740e4735cf1c63d800c243fe920c518662374" Sep 07 17:28:34.474267 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:34.474251 2563 scope.go:117] "RemoveContainer" containerID="221bfd6cb1311d43f9977363e00d0c5eb02b4951721e96f00f5d796bc3c59f2c" Sep 07 17:28:36.426039 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:36.426009 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" path="/var/lib/kubelet/pods/4bc85ce8-a665-466e-87ff-cdb2e68013c0/volumes" Sep 07 17:28:36.426542 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:36.426528 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" path="/var/lib/kubelet/pods/ed67eb91-db64-4602-b08e-3bd01b2a379a/volumes" Sep 07 17:28:42.816961 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:42.816924 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:42.917465 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:42.917428 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:28:43.685987 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:28:43.685954 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:11.897662 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:11.897619 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:11.987995 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:11.987963 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:12.292607 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:12.292574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:12.429636 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:12.429602 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:12.555377 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:12.555301 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:12.656835 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:12.656806 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:12.755474 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:12.755434 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:30:35.208897 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:30:35.208813 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:31:36.851552 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:31:36.851513 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:31:36.951997 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:31:36.951962 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:31:37.281108 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:31:37.281067 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:31:37.379933 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:31:37.379896 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:31:37.584141 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:31:37.584051 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:31:37.718138 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:31:37.718107 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:32:24.097023 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:32:24.096984 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:32:58.680963 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:32:58.680923 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:32:58.775004 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:32:58.774974 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:33:19.844576 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:33:19.844521 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:33:21.875402 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:33:21.875366 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:33:25.161776 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:33:25.161741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:33:27.931643 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:33:27.931608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:33:28.002358 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:33:28.002323 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:01.022808 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:01.022769 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:03.232088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:03.232054 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:03.345432 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:03.345390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:09.473572 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:09.473529 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:09.541549 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:09.541514 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:09.591807 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:09.591775 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd"] Sep 07 17:34:09.592001 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:09.591979 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" podUID="5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f" containerName="manager" containerID="cri-o://cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e" gracePeriod=10 Sep 07 17:34:09.834685 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:09.834652 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:34:09.950376 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:09.950340 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psp95\" (UniqueName: \"kubernetes.io/projected/5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f-kube-api-access-psp95\") pod \"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f\" (UID: \"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f\") " Sep 07 17:34:09.952341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:09.952315 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f-kube-api-access-psp95" (OuterVolumeSpecName: "kube-api-access-psp95") pod "5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f" (UID: "5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f"). InnerVolumeSpecName "kube-api-access-psp95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:34:10.051042 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.050963 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psp95\" (UniqueName: \"kubernetes.io/projected/5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f-kube-api-access-psp95\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:34:10.642192 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.642159 2563 generic.go:356] "Generic (PLEG): container finished" podID="5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f" containerID="cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e" exitCode=0 Sep 07 17:34:10.642671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.642208 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" Sep 07 17:34:10.642671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.642218 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" event={"ID":"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f","Type":"ContainerDied","Data":"cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e"} Sep 07 17:34:10.642671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.642282 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd" event={"ID":"5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f","Type":"ContainerDied","Data":"4c91005995d3f1537cb8073ad1ef8b7459a9728ce72f8495290b6f6ba3e2bc33"} Sep 07 17:34:10.642671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.642304 2563 scope.go:117] "RemoveContainer" containerID="cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e" Sep 07 17:34:10.650124 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.650108 2563 scope.go:117] "RemoveContainer" containerID="cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e" Sep 07 17:34:10.650396 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:34:10.650377 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e\": container with ID starting with cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e not found: ID does not exist" containerID="cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e" Sep 07 17:34:10.650462 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.650406 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e"} err="failed to get container status \"cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e\": rpc error: code = NotFound desc = could not find container \"cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e\": container with ID starting with cb777c6ec0b5520f8edfed30d31e2d090861bbc4572b920af808610c3c99fe6e not found: ID does not exist" Sep 07 17:34:10.676461 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.676425 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd"] Sep 07 17:34:10.689251 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:10.689180 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5xxjd"] Sep 07 17:34:12.426382 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:12.426346 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f" path="/var/lib/kubelet/pods/5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f/volumes" Sep 07 17:34:14.960800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.960760 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k"] Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961161 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="feast-apply" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961179 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="feast-apply" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961190 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961200 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961222 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="feast-apply" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961232 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="feast-apply" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961241 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="feast-init" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961249 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="feast-init" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961263 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961271 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961288 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f" containerName="manager" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961296 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f" containerName="manager" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961308 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961316 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" Sep 07 17:34:14.961319 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961325 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="feast-init" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961333 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="feast-init" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961342 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961348 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961433 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="online" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961445 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="online" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961459 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5912794e-98f1-4fb4-9b40-f4aa9e0f0f2f" containerName="manager" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961469 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc85ce8-a665-466e-87ff-cdb2e68013c0" containerName="registry" Sep 07 17:34:14.962088 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.961479 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed67eb91-db64-4602-b08e-3bd01b2a379a" containerName="registry" Sep 07 17:34:14.964571 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.964552 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" Sep 07 17:34:14.968658 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.968628 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 07 17:34:14.968788 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.968712 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 07 17:34:14.969108 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.969094 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-ntq8g\"" Sep 07 17:34:14.978953 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:14.978930 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k"] Sep 07 17:34:15.090704 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.090664 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scm98\" (UniqueName: \"kubernetes.io/projected/91e0ee05-a17a-47cb-bd2a-a324431632eb-kube-api-access-scm98\") pod \"feast-operator-controller-manager-689b877b4b-5pq4k\" (UID: \"91e0ee05-a17a-47cb-bd2a-a324431632eb\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" Sep 07 17:34:15.191592 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.191552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scm98\" (UniqueName: \"kubernetes.io/projected/91e0ee05-a17a-47cb-bd2a-a324431632eb-kube-api-access-scm98\") pod \"feast-operator-controller-manager-689b877b4b-5pq4k\" (UID: \"91e0ee05-a17a-47cb-bd2a-a324431632eb\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" Sep 07 17:34:15.201256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.201224 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scm98\" (UniqueName: \"kubernetes.io/projected/91e0ee05-a17a-47cb-bd2a-a324431632eb-kube-api-access-scm98\") pod \"feast-operator-controller-manager-689b877b4b-5pq4k\" (UID: \"91e0ee05-a17a-47cb-bd2a-a324431632eb\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" Sep 07 17:34:15.273958 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.273925 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" Sep 07 17:34:15.547508 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.547430 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k"] Sep 07 17:34:15.555906 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:34:15.555876 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e0ee05_a17a_47cb_bd2a_a324431632eb.slice/crio-2d8a00e3a96686da941f774fc04484205d880ad1c7215e2d0d359b637985e2dc WatchSource:0}: Error finding container 2d8a00e3a96686da941f774fc04484205d880ad1c7215e2d0d359b637985e2dc: Status 404 returned error can't find the container with id 2d8a00e3a96686da941f774fc04484205d880ad1c7215e2d0d359b637985e2dc Sep 07 17:34:15.662393 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.662358 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" event={"ID":"91e0ee05-a17a-47cb-bd2a-a324431632eb","Type":"ContainerStarted","Data":"4863932cd8ff23d5cd191c235d3ee368ef4a5aaff4aeb30093435ed603eb2616"} Sep 07 17:34:15.662545 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.662413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" event={"ID":"91e0ee05-a17a-47cb-bd2a-a324431632eb","Type":"ContainerStarted","Data":"2d8a00e3a96686da941f774fc04484205d880ad1c7215e2d0d359b637985e2dc"} Sep 07 17:34:15.662545 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.662463 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" Sep 07 17:34:15.680174 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:15.680126 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" podStartSLOduration=1.6801096009999998 podStartE2EDuration="1.680109601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:34:15.677227473 +0000 UTC m=+1563.750128805" watchObservedRunningTime="2026-09-07 17:34:15.680109601 +0000 UTC m=+1563.753010934" Sep 07 17:34:26.668477 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:26.668446 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pq4k" Sep 07 17:34:29.276514 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:29.276472 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:29.376785 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:29.376750 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:32.316000 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:32.315958 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:32.417548 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:32.417498 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:32.517800 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:32.517760 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:32.650400 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:32.650299 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:34:32.869492 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:34:32.869460 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:35:56.363047 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:35:56.363008 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:35:56.460116 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:35:56.460079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:35:56.678591 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:35:56.678509 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:35:56.780341 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:35:56.780304 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:35:56.882850 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:35:56.882823 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:35:56.959389 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:35:56.959307 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:37:21.964816 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:37:21.964771 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:37:22.026008 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:37:22.025971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:05.628412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:05.628363 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:07.012264 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:07.012225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:10.912191 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:10.912146 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:48.598001 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:48.597917 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:48.705688 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:48.705639 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:54.290172 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.290128 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:54.353809 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.353773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:54.666836 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.666754 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w47ts/must-gather-vdlcp"] Sep 07 17:39:54.670130 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.670108 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:54.672020 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.671997 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w47ts\"/\"kube-root-ca.crt\"" Sep 07 17:39:54.672366 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.672351 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w47ts\"/\"openshift-service-ca.crt\"" Sep 07 17:39:54.672416 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.672370 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w47ts\"/\"default-dockercfg-m6vgs\"" Sep 07 17:39:54.679355 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.679321 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:54.679668 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.679649 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w47ts/must-gather-vdlcp"] Sep 07 17:39:54.740612 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.740576 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:39:54.846980 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.846942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87ltm\" (UniqueName: \"kubernetes.io/projected/619c8830-a30e-4108-8e1a-b70b047aa29d-kube-api-access-87ltm\") pod \"must-gather-vdlcp\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:54.846980 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.846982 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/619c8830-a30e-4108-8e1a-b70b047aa29d-must-gather-output\") pod \"must-gather-vdlcp\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:54.947808 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.947696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ltm\" (UniqueName: \"kubernetes.io/projected/619c8830-a30e-4108-8e1a-b70b047aa29d-kube-api-access-87ltm\") pod \"must-gather-vdlcp\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:54.947808 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.947763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/619c8830-a30e-4108-8e1a-b70b047aa29d-must-gather-output\") pod \"must-gather-vdlcp\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:54.948090 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.948075 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/619c8830-a30e-4108-8e1a-b70b047aa29d-must-gather-output\") pod \"must-gather-vdlcp\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:54.956359 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.956331 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87ltm\" (UniqueName: \"kubernetes.io/projected/619c8830-a30e-4108-8e1a-b70b047aa29d-kube-api-access-87ltm\") pod \"must-gather-vdlcp\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:54.983992 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:54.983951 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:39:55.255311 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:55.255190 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w47ts/must-gather-vdlcp"] Sep 07 17:39:55.257995 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:39:55.257967 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod619c8830_a30e_4108_8e1a_b70b047aa29d.slice/crio-eb9774180da5eb78442aaaab7adba1892807e09b2c8a3e6d10463c5a1e65e638 WatchSource:0}: Error finding container eb9774180da5eb78442aaaab7adba1892807e09b2c8a3e6d10463c5a1e65e638: Status 404 returned error can't find the container with id eb9774180da5eb78442aaaab7adba1892807e09b2c8a3e6d10463c5a1e65e638 Sep 07 17:39:55.259862 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:55.259845 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 17:39:55.888968 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:39:55.888929 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w47ts/must-gather-vdlcp" event={"ID":"619c8830-a30e-4108-8e1a-b70b047aa29d","Type":"ContainerStarted","Data":"eb9774180da5eb78442aaaab7adba1892807e09b2c8a3e6d10463c5a1e65e638"} Sep 07 17:40:01.918392 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:01.918306 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w47ts/must-gather-vdlcp" event={"ID":"619c8830-a30e-4108-8e1a-b70b047aa29d","Type":"ContainerStarted","Data":"a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8"} Sep 07 17:40:01.918392 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:01.918364 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w47ts/must-gather-vdlcp" event={"ID":"619c8830-a30e-4108-8e1a-b70b047aa29d","Type":"ContainerStarted","Data":"44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65"} Sep 07 17:40:01.933358 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:01.933236 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-w47ts/must-gather-vdlcp" podStartSLOduration=1.791644411 podStartE2EDuration="7.933213583s" totalImagesPullingTime="6.141569172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:40:01.932877273 +0000 UTC m=+1910.005778607" watchObservedRunningTime="2026-09-07 17:40:01.933213583 +0000 UTC m=+1910.006114917" Sep 07 17:40:11.784373 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:11.784333 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:40:11.849198 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:11.845210 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:40:11.960069 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:11.960034 2563 generic.go:356] "Generic (PLEG): container finished" podID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerID="44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65" exitCode=0 Sep 07 17:40:11.960250 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:11.960084 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w47ts/must-gather-vdlcp" event={"ID":"619c8830-a30e-4108-8e1a-b70b047aa29d","Type":"ContainerDied","Data":"44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65"} Sep 07 17:40:11.960427 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:11.960412 2563 scope.go:117] "RemoveContainer" containerID="44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65" Sep 07 17:40:12.216279 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.216188 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-w47ts_must-gather-vdlcp_619c8830-a30e-4108-8e1a-b70b047aa29d/gather/0.log" Sep 07 17:40:12.557233 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.557196 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:40:12.680069 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.680031 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mvt4j/must-gather-xffzm"] Sep 07 17:40:12.682347 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.682330 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:12.684222 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.684161 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mvt4j\"/\"openshift-service-ca.crt\"" Sep 07 17:40:12.684426 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.684160 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mvt4j\"/\"default-dockercfg-8v5b4\"" Sep 07 17:40:12.684495 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.684193 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mvt4j\"/\"kube-root-ca.crt\"" Sep 07 17:40:12.691507 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.691475 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvt4j/must-gather-xffzm"] Sep 07 17:40:12.706211 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.706172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:40:12.770971 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.770918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:40:12.806269 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.806228 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg9s5\" (UniqueName: \"kubernetes.io/projected/213a9e2c-36a2-4493-bd07-b3c0337dabf3-kube-api-access-kg9s5\") pod \"must-gather-xffzm\" (UID: \"213a9e2c-36a2-4493-bd07-b3c0337dabf3\") " pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:12.806633 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.806273 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/213a9e2c-36a2-4493-bd07-b3c0337dabf3-must-gather-output\") pod \"must-gather-xffzm\" (UID: \"213a9e2c-36a2-4493-bd07-b3c0337dabf3\") " pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:12.906851 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.906761 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg9s5\" (UniqueName: \"kubernetes.io/projected/213a9e2c-36a2-4493-bd07-b3c0337dabf3-kube-api-access-kg9s5\") pod \"must-gather-xffzm\" (UID: \"213a9e2c-36a2-4493-bd07-b3c0337dabf3\") " pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:12.906851 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.906798 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/213a9e2c-36a2-4493-bd07-b3c0337dabf3-must-gather-output\") pod \"must-gather-xffzm\" (UID: \"213a9e2c-36a2-4493-bd07-b3c0337dabf3\") " pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:12.907210 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.907179 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/213a9e2c-36a2-4493-bd07-b3c0337dabf3-must-gather-output\") pod \"must-gather-xffzm\" (UID: \"213a9e2c-36a2-4493-bd07-b3c0337dabf3\") " pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:12.914077 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.914055 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg9s5\" (UniqueName: \"kubernetes.io/projected/213a9e2c-36a2-4493-bd07-b3c0337dabf3-kube-api-access-kg9s5\") pod \"must-gather-xffzm\" (UID: \"213a9e2c-36a2-4493-bd07-b3c0337dabf3\") " pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:12.992570 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:12.992530 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvt4j/must-gather-xffzm" Sep 07 17:40:13.264116 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:13.264089 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvt4j/must-gather-xffzm"] Sep 07 17:40:13.266513 ip-10-0-143-85 kubenswrapper[2563]: W0907 17:40:13.266473 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod213a9e2c_36a2_4493_bd07_b3c0337dabf3.slice/crio-e531022a235f9a21c61d90b9b946226e023f2f83d30a07363022b1938ea0a068 WatchSource:0}: Error finding container e531022a235f9a21c61d90b9b946226e023f2f83d30a07363022b1938ea0a068: Status 404 returned error can't find the container with id e531022a235f9a21c61d90b9b946226e023f2f83d30a07363022b1938ea0a068 Sep 07 17:40:13.969669 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:13.969625 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvt4j/must-gather-xffzm" event={"ID":"213a9e2c-36a2-4493-bd07-b3c0337dabf3","Type":"ContainerStarted","Data":"e531022a235f9a21c61d90b9b946226e023f2f83d30a07363022b1938ea0a068"} Sep 07 17:40:14.982538 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:14.982494 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvt4j/must-gather-xffzm" event={"ID":"213a9e2c-36a2-4493-bd07-b3c0337dabf3","Type":"ContainerStarted","Data":"30d38f6a78a423ed270499b5b84a0b8d9acb5910761c1b7ef0bc0bc6038b5692"} Sep 07 17:40:14.983413 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:14.983387 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvt4j/must-gather-xffzm" event={"ID":"213a9e2c-36a2-4493-bd07-b3c0337dabf3","Type":"ContainerStarted","Data":"33be32a035ae8ba32d9be989832d3bf1eb7439cf8173de773f95c140976c444f"} Sep 07 17:40:14.998582 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:14.998524 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mvt4j/must-gather-xffzm" podStartSLOduration=2.248872612 podStartE2EDuration="2.998505538s" totalImagesPullingTime="749.632926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 17:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:40:14.998114403 +0000 UTC m=+1923.071015732" watchObservedRunningTime="2026-09-07 17:40:14.998505538 +0000 UTC m=+1923.071406871" Sep 07 17:40:15.392381 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:15.392347 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nmmvg_d40dd6a8-670a-447b-9d18-7ec8645d13ef/global-pull-secret-syncer/0.log" Sep 07 17:40:15.503683 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:15.503653 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" Sep 07 17:40:15.641985 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:15.641950 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-85.ec2.internal_f5d8b8706f17884ecb94f9bab542e98f/haproxy/0.log" Sep 07 17:40:17.532549 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.532509 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-w47ts/must-gather-vdlcp"] Sep 07 17:40:17.533159 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.532786 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-w47ts/must-gather-vdlcp" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerName="copy" containerID="cri-o://a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8" gracePeriod=2 Sep 07 17:40:17.537377 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.537315 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-w47ts/must-gather-vdlcp"] Sep 07 17:40:17.910757 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.904341 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-w47ts_must-gather-vdlcp_619c8830-a30e-4108-8e1a-b70b047aa29d/copy/0.log" Sep 07 17:40:17.910757 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.908686 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:40:17.911885 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:40:17.911831 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-vdlcp\" is forbidden: User \"system:node:ip-10-0-143-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-w47ts\": no relationship found between node 'ip-10-0-143-85.ec2.internal' and this object" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:40:17.958081 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.957162 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87ltm\" (UniqueName: \"kubernetes.io/projected/619c8830-a30e-4108-8e1a-b70b047aa29d-kube-api-access-87ltm\") pod \"619c8830-a30e-4108-8e1a-b70b047aa29d\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " Sep 07 17:40:17.958081 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.957228 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/619c8830-a30e-4108-8e1a-b70b047aa29d-must-gather-output\") pod \"619c8830-a30e-4108-8e1a-b70b047aa29d\" (UID: \"619c8830-a30e-4108-8e1a-b70b047aa29d\") " Sep 07 17:40:17.958081 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.957783 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/619c8830-a30e-4108-8e1a-b70b047aa29d-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "619c8830-a30e-4108-8e1a-b70b047aa29d" (UID: "619c8830-a30e-4108-8e1a-b70b047aa29d"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 17:40:17.970700 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:17.970655 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619c8830-a30e-4108-8e1a-b70b047aa29d-kube-api-access-87ltm" (OuterVolumeSpecName: "kube-api-access-87ltm") pod "619c8830-a30e-4108-8e1a-b70b047aa29d" (UID: "619c8830-a30e-4108-8e1a-b70b047aa29d"). InnerVolumeSpecName "kube-api-access-87ltm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 17:40:18.000766 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.000661 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-w47ts_must-gather-vdlcp_619c8830-a30e-4108-8e1a-b70b047aa29d/copy/0.log" Sep 07 17:40:18.004117 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.001295 2563 generic.go:356] "Generic (PLEG): container finished" podID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerID="a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8" exitCode=143 Sep 07 17:40:18.004117 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.001419 2563 scope.go:117] "RemoveContainer" containerID="a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8" Sep 07 17:40:18.004117 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.001581 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:40:18.017978 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:40:18.017921 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-vdlcp\" is forbidden: User \"system:node:ip-10-0-143-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-w47ts\": no relationship found between node 'ip-10-0-143-85.ec2.internal' and this object" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:40:18.027961 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:40:18.027912 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-vdlcp\" is forbidden: User \"system:node:ip-10-0-143-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-w47ts\": no relationship found between node 'ip-10-0-143-85.ec2.internal' and this object" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" pod="openshift-must-gather-w47ts/must-gather-vdlcp" Sep 07 17:40:18.033799 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.033769 2563 scope.go:117] "RemoveContainer" containerID="44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65" Sep 07 17:40:18.058296 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.058228 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87ltm\" (UniqueName: \"kubernetes.io/projected/619c8830-a30e-4108-8e1a-b70b047aa29d-kube-api-access-87ltm\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:40:18.058296 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.058264 2563 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/619c8830-a30e-4108-8e1a-b70b047aa29d-must-gather-output\") on node \"ip-10-0-143-85.ec2.internal\" DevicePath \"\"" Sep 07 17:40:18.067579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.066644 2563 scope.go:117] "RemoveContainer" containerID="a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8" Sep 07 17:40:18.067579 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:40:18.067077 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8\": container with ID starting with a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8 not found: ID does not exist" containerID="a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8" Sep 07 17:40:18.067579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.067117 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8"} err="failed to get container status \"a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8\": rpc error: code = NotFound desc = could not find container \"a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8\": container with ID starting with a09d08f678535bb491c074321a7996ccc40e97bd1ccc0a067ab8c247ca82c1b8 not found: ID does not exist" Sep 07 17:40:18.067579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.067144 2563 scope.go:117] "RemoveContainer" containerID="44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65" Sep 07 17:40:18.067579 ip-10-0-143-85 kubenswrapper[2563]: E0907 17:40:18.067398 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65\": container with ID starting with 44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65 not found: ID does not exist" containerID="44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65" Sep 07 17:40:18.067579 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.067421 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65"} err="failed to get container status \"44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65\": rpc error: code = NotFound desc = could not find container \"44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65\": container with ID starting with 44c69665282dd312d0ed3052885c4e740975486cc1bbe740ed6e78f14c7a7f65 not found: ID does not exist" Sep 07 17:40:18.081870 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.081737 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:40:18.139862 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.139652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ngms_9e168995-8f48-4329-b17f-e1f39943efc8/konnectivity-agent/0.log" limit=1024 Sep 07 17:40:18.429155 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:18.429070 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" path="/var/lib/kubelet/pods/619c8830-a30e-4108-8e1a-b70b047aa29d/volumes" Sep 07 17:40:19.621678 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.621648 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_32c5aaaf-93f2-4b41-97da-b4f00a6df3b7/alertmanager/0.log" Sep 07 17:40:19.650393 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.650337 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_32c5aaaf-93f2-4b41-97da-b4f00a6df3b7/config-reloader/0.log" Sep 07 17:40:19.679673 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.679649 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_32c5aaaf-93f2-4b41-97da-b4f00a6df3b7/kube-rbac-proxy-web/0.log" Sep 07 17:40:19.706234 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.706210 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_32c5aaaf-93f2-4b41-97da-b4f00a6df3b7/kube-rbac-proxy/0.log" Sep 07 17:40:19.736464 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.736430 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_32c5aaaf-93f2-4b41-97da-b4f00a6df3b7/kube-rbac-proxy-metric/0.log" Sep 07 17:40:19.761307 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.761266 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_32c5aaaf-93f2-4b41-97da-b4f00a6df3b7/prom-label-proxy/0.log" Sep 07 17:40:19.783085 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.783039 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_32c5aaaf-93f2-4b41-97da-b4f00a6df3b7/init-config-reloader/0.log" Sep 07 17:40:19.845552 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.845519 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hf5lk_55c4ac9c-51d7-4cb9-aa23-6f9c45c83748/kube-state-metrics/0.log" Sep 07 17:40:19.866112 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.866081 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hf5lk_55c4ac9c-51d7-4cb9-aa23-6f9c45c83748/kube-rbac-proxy-main/0.log" Sep 07 17:40:19.888707 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.888633 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hf5lk_55c4ac9c-51d7-4cb9-aa23-6f9c45c83748/kube-rbac-proxy-self/0.log" Sep 07 17:40:19.919586 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.919558 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64868b84df-pqxn4_8313e248-1c50-459e-b0fb-c7743aa20db0/metrics-server/0.log" Sep 07 17:40:19.980186 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.980152 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ndnfg_2e2f8587-20ab-4545-b59a-7efe60c32af2/node-exporter/0.log" Sep 07 17:40:19.999745 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:19.999702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ndnfg_2e2f8587-20ab-4545-b59a-7efe60c32af2/kube-rbac-proxy/0.log" Sep 07 17:40:20.023223 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:20.023195 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ndnfg_2e2f8587-20ab-4545-b59a-7efe60c32af2/init-textfile/0.log" Sep 07 17:40:20.446534 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:20.446483 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-gn26v_603fcde4-00ab-40ee-b6cd-219c4d4b9a68/prometheus-operator/0.log" Sep 07 17:40:20.482851 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:20.482804 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-gn26v_603fcde4-00ab-40ee-b6cd-219c4d4b9a68/kube-rbac-proxy/0.log" Sep 07 17:40:20.519299 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:20.519264 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-bv47x_1a7b104d-1b06-4a8f-b756-24b8f59fa1de/prometheus-operator-admission-webhook/0.log" Sep 07 17:40:20.574240 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:20.574206 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d754ffdc-wr4dx_a7635e9f-0653-45a3-9929-436bc8593e15/telemeter-client/0.log" Sep 07 17:40:20.611746 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:20.611694 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d754ffdc-wr4dx_a7635e9f-0653-45a3-9929-436bc8593e15/reload/0.log" Sep 07 17:40:20.651338 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:20.651284 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d754ffdc-wr4dx_a7635e9f-0653-45a3-9929-436bc8593e15/kube-rbac-proxy/0.log" Sep 07 17:40:22.621849 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.621813 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8"] Sep 07 17:40:22.622418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.622205 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerName="copy" Sep 07 17:40:22.622418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.622220 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerName="copy" Sep 07 17:40:22.622418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.622255 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerName="gather" Sep 07 17:40:22.622418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.622265 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerName="gather" Sep 07 17:40:22.622418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.622343 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerName="gather" Sep 07 17:40:22.622418 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.622357 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="619c8830-a30e-4108-8e1a-b70b047aa29d" containerName="copy" Sep 07 17:40:22.627872 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.627840 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.638707 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.638675 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8"] Sep 07 17:40:22.707619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.707374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-proc\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.707619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.707426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf9cv\" (UniqueName: \"kubernetes.io/projected/813063a1-77dc-4a77-b970-08d43bceb980-kube-api-access-cf9cv\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.707619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.707478 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-sys\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.707619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.707503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-podres\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.707619 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.707533 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-lib-modules\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-sys\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808412 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-podres\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-lib-modules\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808496 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-sys\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808565 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-proc\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808596 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-podres\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf9cv\" (UniqueName: \"kubernetes.io/projected/813063a1-77dc-4a77-b970-08d43bceb980-kube-api-access-cf9cv\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808671 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808622 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-proc\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.808915 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.808675 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/813063a1-77dc-4a77-b970-08d43bceb980-lib-modules\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.815942 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.815911 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf9cv\" (UniqueName: \"kubernetes.io/projected/813063a1-77dc-4a77-b970-08d43bceb980-kube-api-access-cf9cv\") pod \"perf-node-gather-daemonset-hpdt8\" (UID: \"813063a1-77dc-4a77-b970-08d43bceb980\") " pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:22.824483 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.824457 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77b4ff89f6-ssg5g_ae2bcfbe-52f2-4a5d-b07f-4e5b77870d7c/console/0.log" Sep 07 17:40:22.942250 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:22.942168 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:23.220057 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:23.219913 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8"] Sep 07 17:40:24.038287 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.038009 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" event={"ID":"813063a1-77dc-4a77-b970-08d43bceb980","Type":"ContainerStarted","Data":"5c35b95627c0d389c641e6405244dd6e1273908f55fbd0a5faa8f1eeddec1455"} Sep 07 17:40:24.038287 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.038080 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" event={"ID":"813063a1-77dc-4a77-b970-08d43bceb980","Type":"ContainerStarted","Data":"a3f65d2d78e673d3d5e4d7b1e6d3d5fa44d06a8b78fc1888b367f98ca2a4f531"} Sep 07 17:40:24.038845 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.038332 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:24.055576 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.055523 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" podStartSLOduration=2.055503856 podStartE2EDuration="2.055503856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 17:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 17:40:24.053996837 +0000 UTC m=+1932.126898183" watchObservedRunningTime="2026-09-07 17:40:24.055503856 +0000 UTC m=+1932.128405189" Sep 07 17:40:24.105528 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.105488 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2tgv4_2d2c8366-312c-4f0d-929b-ad4245f7c236/dns/0.log" Sep 07 17:40:24.124256 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.124226 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2tgv4_2d2c8366-312c-4f0d-929b-ad4245f7c236/kube-rbac-proxy/0.log" Sep 07 17:40:24.277326 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.277296 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r97tb_36380f08-07a9-4f59-9a84-8a639a5943cd/dns-node-resolver/0.log" Sep 07 17:40:24.710100 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:24.710069 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bgsdc_02394e61-3140-406c-afa3-84ffc266a641/node-ca/0.log" Sep 07 17:40:25.705019 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:25.704991 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2l6hx_d3b8ac2d-79b3-4163-898f-aa94677f513b/serve-healthcheck-canary/0.log" Sep 07 17:40:26.175479 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:26.175450 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hk99l_d45929c7-ef4e-4f12-b694-52cb7b8eb013/kube-rbac-proxy/0.log" Sep 07 17:40:26.195916 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:26.195889 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hk99l_d45929c7-ef4e-4f12-b694-52cb7b8eb013/exporter/0.log" Sep 07 17:40:26.217408 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:26.217366 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hk99l_d45929c7-ef4e-4f12-b694-52cb7b8eb013/extractor/0.log" Sep 07 17:40:30.056058 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:30.055372 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mvt4j/perf-node-gather-daemonset-hpdt8" Sep 07 17:40:31.572679 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:31.572655 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz454_c9a30523-229a-47bb-8e6e-c7f2b20fd301/kube-multus-additional-cni-plugins/0.log" Sep 07 17:40:31.592906 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:31.592869 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz454_c9a30523-229a-47bb-8e6e-c7f2b20fd301/egress-router-binary-copy/0.log" Sep 07 17:40:31.613356 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:31.613327 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz454_c9a30523-229a-47bb-8e6e-c7f2b20fd301/cni-plugins/0.log" Sep 07 17:40:31.632384 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:31.632355 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz454_c9a30523-229a-47bb-8e6e-c7f2b20fd301/bond-cni-plugin/0.log" Sep 07 17:40:31.652808 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:31.652772 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz454_c9a30523-229a-47bb-8e6e-c7f2b20fd301/routeoverride-cni/0.log" Sep 07 17:40:31.674484 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:31.674456 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz454_c9a30523-229a-47bb-8e6e-c7f2b20fd301/whereabouts-cni-bincopy/0.log" Sep 07 17:40:31.698064 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:31.698034 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz454_c9a30523-229a-47bb-8e6e-c7f2b20fd301/whereabouts-cni/0.log" Sep 07 17:40:32.169277 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:32.169196 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zgbmj_96f76600-5d34-4109-8f07-3b7fa97658a4/kube-multus/0.log" Sep 07 17:40:32.299590 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:32.299557 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cx8hf_d590b05d-a3ad-434d-bc55-eb1f23c0e214/network-metrics-daemon/0.log" Sep 07 17:40:32.320394 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:32.320363 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cx8hf_d590b05d-a3ad-434d-bc55-eb1f23c0e214/kube-rbac-proxy/0.log" Sep 07 17:40:33.114145 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.114117 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/ovn-controller/0.log" Sep 07 17:40:33.132239 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.132205 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/ovn-acl-logging/0.log" Sep 07 17:40:33.156602 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.156570 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/kube-rbac-proxy-node/0.log" Sep 07 17:40:33.179281 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.179256 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/kube-rbac-proxy-ovn-metrics/0.log" Sep 07 17:40:33.195821 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.195786 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/northd/0.log" Sep 07 17:40:33.215833 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.215806 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/nbdb/0.log" Sep 07 17:40:33.234327 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.234301 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/sbdb/0.log" Sep 07 17:40:33.291530 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:33.291496 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfwh8_b29aaeeb-cf84-4293-b329-32de1e621742/ovnkube-controller/0.log" Sep 07 17:40:34.707834 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:34.707807 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-vmndt_4cc3bdcb-b505-48e8-9794-63d6117a1163/network-check-target-container/0.log" Sep 07 17:40:35.605221 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:35.605192 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-ps5c8_66832598-bf55-4875-9c98-0b96b2582b7c/iptables-alerter/0.log" Sep 07 17:40:36.253105 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:36.253078 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-nxwnb_6e09d3ea-b5e9-4f78-be5d-c9ec9258d497/tuned/0.log" Sep 07 17:40:39.099005 ip-10-0-143-85 kubenswrapper[2563]: I0907 17:40:39.098950 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-75d99c484-9r6dk_a1549cc9-1b91-4990-b573-645b117a4878/service-ca-controller/0.log"