Based on the theory of heat transfer, the influence of expansion joints on the temperature and stress distribution of ladle lining is discussed. In view of the current expansion joint, the mathematical model of heat transfer and the three dimensional finite element model of ladle lining brick are established. By analyzing the temperature and stress distribution of ladle lining brick when the expansion joints are in different sizes, the thermal mechanical stress caused by the severe temperature difference can be reduced by the suitable expansion joint of the lining brick during the ladle baking and working process. The analysis results showed that the thermal mechanical stress which is caused by thermal expansion can be released through the 2 mm expansion joint, which is set in the building process. So we can effectively reduce the thermal mechanical stress of the ladle lining, and there is no risk of steel leakage, thus the service life of ladle can be effectively prolonged.