The organization of production factors and the advancement of agricultural technology are crucial elements for promoting the high-quality development of agriculture and achieving the comprehensive revitalization of rural areas. Matching the structure of factor input and the direction of technological progress to improve ATFP has become a key issue. Based on the input structure of agricultural production and the standardized CES production function, this paper derives the ATFP function, which incorporates a biased technological progress index and capital deepening degree, and systematically describes the influence mechanism of biased technological progress, capital deepening, and factor efficiency on ATFP. The results show that, compared with the base period, both the level of capital deepening and the technological progress index of capital bias in China’s agricultural sector have increased significantly. The relationship between capital deepening and technological progress bias will significantly impact the size and direction of their interaction effect. This interaction effect, in turn, will further influence the growth of ATFP and may have a positive or negative impact on ATFP. This effect will also vary due to differences in factor input structure and factor efficiency between provinces and major agricultural producing areas. At the same time, this paper also elucidates the contradiction between the shift in agricultural factor input structure and the relative stability of ATFP in China since 1985, offering insights for the future development of agriculture and the enhancement of ATFP within the existing agricultural factor input framework.