Improved understanding of soil fertility factors limiting crop productivity is important to develop appropriate soil and nutrient management recommendations in sub-Saharan Africa. Diagnostic trials were implemented in Kenya, Malawi, Mali, Nigeria and Tanzania, as part of the African Soils Information Service (AfSIS) project, to identify soil fertility constraints to crop production across various cropping systems and soil fertility conditions. In each country, one to three sites of 10km×10km were included with each site having 12–31 field trials. The treatments tested included a control, an NPK treatment, three treatments in which the N, P and K nutrients were omitted one at a time from the NPK treatment, and three treatments in which secondary and micronutrients (Ca, Mg, S, Zn and B) simply referred here as multi-nutrients, manure and lime were added to the NPK. The field trials were conducted for 1–2 seasons; the test crop was maize except in Mali where sorghum was used. Nitrogen was limiting in all sites and generally the most limiting nutrient except in Sidindi (Kenya) and Kontela (Mali) where P was the most limiting. The general pattern in Kiberashi (Tanzania) shows none of the nutrients were limiting. K is mainly limiting in only one site (Mbinga) although incidences of K limitation were seen in almost all sites. Addition of multi-nutrients and manure further improved the yields of NPK in most sites. Cluster analyses revealed that maize crop in 11% of fields were highly responsive to nitrogen application, 25% (i.e., 21% poor and 4% fertile) ‘non-responsive’ to any nutrient or soil amendment, 28% being ‘low responsive’ and 36% of ‘intermediate response’. This study indicates that constraints to crop production vary considerably even within a site, and that addressing limitations in secondary and micronutrients, and increasing soil carbon can improve response to fertilizers. For sustainable crop production intensification in smallholder farming systems in SSA, there is need to develop management strategies to improve efficiency of fertilizer use and of other inputs, recognizing the site-specific nutrient response patterns at various spatial scales.