In order to solve the nonsingular and ill conditioned problems of the coefficient matrix of linear equations and the slow calculation speed when solving incremental equations in Bundle Adjustment optimization process, an improved gradient descent algorithm based on Gauss Newton algorithm and Levenberg Marquardt algorithm is proposed. The algorithm mainly finds the best increment by adding a dynamic trust region to the increment. In this effective region, the increment is calculated by the size of Lagrange multiplier to find the best point in the region. The global consistency and real‐time performance of slam are improved by optimizing the camera pose and feature points of adjacent key frames in real time. The algorithm avoids the nonsingular and ill conditioned problems of the coefficient matrix of linear equations to a certain extent, corrects the stability problem of Gauss Newton algorithm, and improves the calculation speed of Levenberg Marquardt method. Experimental results based on datasets and real scenes show that the performance of the algorithm is better than the mainstream algorithms such as Gauss Newton algorithm and Levenberg Marquardt algorithm in many real scenes.