Bishop McTyeire was born 28 July 1824 in Barnwell County, South Carolina, the son of Methodist parents. Holland converted to Christianity at an early age, professing his faith at thirteen. He attended the higher schools available at the time: first at Cokesbury, South Carolina, then Collinsworth Institute in Georgia. By these schools he prepared himself for college.

Holland graduated from Randolph-Macon College in Virginia (A.B. degree, 1844). He was among the earliest graduates of this institution, the mother college of Southern Methodism. One biographer states of McTyeire, "[Randolph-Macon] has given to the Church and world no son of greater worth and distinction."

Already licensed to preach, McTyeire was admitted on trial into the Virginia Annual Conference in November 1845. He was appointed to Williamsburg, Virginia. After one year's service, he was transferred to the Alabama Conference, admitted into full connection at the first of 1848. In Alabama he pastored at Mobile and Demopolis. He also pastored in Columbus, Mississippi, transferring to the Louisiana Conference, where he was ordained elder in 1849. He also was a pastor in New Orleans.

In 1854, McTyeire was elected editor of the New Orleans Christian Advocate, serving in this position until 1858. He was then elected editor of the Nashville Christian Advocate, the central organ of the M.E. Church, South. Interrupted in his editorial career by the American Civil War, he entered the pastorate again in the Alabama Conference, serving in the city of Montgomery, from which he was elected to the episcopacy in 1866 at the General Conference meeting that year in New Orleans.

McTyeire led a movement within the Methodist Episcopal Church, South, to establish "an institution of learning of the highest order." In 1872, a charter for a "Central University" was issued to the bishop and fellow petitioners, who represented the nine M.E. Church, South Annual Conferences of the mid-south. Their efforts failed, however, for lack of financial resources.

Early in 1873, McTyeire went to New York for medical treatment. The bishop's wife was a cousin to Commodore Cornelius Vanderbilt's second wife. This connection led to Vanderbilt giving McTyeire two $500,000 gifts, which the bishop used to found Vanderbilt University. The Commodore's gift was given with the understanding that McTyeire would serve as chairman of the university's Board of Trust for life.