The use of high-altitude remote sensing (RS) data from aerial and satellite platforms presents considerable challenges for agricultural monitoring and crop yield estimation due to the presence of noise caused by atmospheric interference, sensor anomalies, and outlier pixel values. This paper introduces a "Quartile Clean Image" pre-processing technique to address these data issues by analyzing quartile pixel values in local neighborhoods to identify and adjust outliers. Applying this technique to 20,946 Moderate Resolution Imaging Spectroradiometer (MODIS) images from 2003 to 2015 improved the mean peak signal-to-noise ratio (PSNR) to 40.91 dB. Integrating Quartile Clean data with Convolutional Neural Networks (CNN) models with exponential decay learning rate scheduling achieved RMSE improvements up to 5.88% for soybeans and 21.85% for corn, while Long Short-Term Memory (LSTM) models demonstrated RMSE reductions up to 11.52% for soybeans and 29.92% for corn using exponential decay learning rates. To compare the proposed method with state-of-the-art techniques, we introduce the Vision Transformer (ViT) model for crop yield estimation. The ViT model, applied to the same dataset, achieves remarkable performance without explicit pre-processing, with R2 scores ranging from 0.9752 to 0.9875 for soybean and 0.9540 to 0.9888 for corn yield estimation. The RMSE values range from 7.75086 to 9.76838 for soybean and 26.25265 to 34.20382 for corn, demonstrating the ViT model's robustness. This research contributes by (1) introducing the Quartile Clean Image method for enhancing RS data quality and improving crop yield estimation accuracy, and (2) comparing it with the state-of-the-art ViT model. The results demonstrate the effectiveness of the proposed approach and highlight the potential of the ViT model for crop yield estimation, representing a valuable advancement in processing high-altitude imagery for precision agriculture applications.