Drought (water deficits) and heat (high temperatures) stress are the prime abiotic constraints, under the current and climate change scenario in future. Any further increase in the occurrence, and extremity of these stresses, either individually or in combination, would severely reduce the crop productivity and food security, globally. Although, they obstruct productivity at all crop growth stages, the extent of damage at reproductive phase of crop growth, mainly the seed filling phase, is critical and causes considerable yield losses. Drought and heat stress substantially affect the seed yields by reducing seed size and number, eventually affecting the commercial trait ‘100 seed weight’ and seed quality. Seed filling is influenced by various metabolic processes occurring in the leaves, especially production and translocation of photoassimilates, importing precursors for biosynthesis of seed reserves, minerals and other functional constituents. These processes are highly sensitive to drought and heat, due to involvement of array of diverse enzymes and transporters, located in the leaves and seeds. We highlight here the findings in various food crops showing how their seed composition is drastically impacted at various cellular levels due to drought and heat stresses, applied separately, or in combination. The combined stresses are extremely detrimental for seed yield and its quality, and thus need more attention. Understanding the precise target sites regulating seed filling events in leaves and seeds, and how they are affected by abiotic stresses, is imperative to enhance the seed quality. It is vital to know the physiological, biochemical and genetic mechanisms, which govern the various seed filling events under stress environments, to devise strategies to improve stress tolerance. Converging modern advances in physiology, biochemistry and biotechnology, especially the “omics” technologies might provide a strong impetus to research on this aspect. Such application, along with effective agronomic management system would pave the way in developing crop genotypes/varieties with improved productivity under drought and/or heat stresses.