The first schoolhouse began in 1872 with 40 students in a single room partitioned off the church.
In 1889 a red brick schoolhouse was built across the street from the church. Until 1942 lay teachers taught at the school that still stands to this day as a private residence.
In 1942, five nuns from the School Sisters of Notre Dame took up residence nearby and took over the running of St. Mary School.
In 1954 the present school was built on Irvine street. It was dedicated by Bishop J.F. Ryan and in 1965 a new room was added.
The old schoolhouse was used as a parish hall until 1974, when it was sold.
In 1967, Elora and Fergus schools were amalgamated. Pupils in Grades One through Three attended St. Joseph School, Fergus and pupils in Kindergarten and Grades Four through Eight attended St. Mary School, Elora.
In 1974 a new addition was built onto St. Mary’s and it included a gymnasium, two classrooms, a staff room, kitchen and storage facilities.
In 1982, with the construction of St. Joseph’s portable school in Fergus all the Fergus students could now attend school there and St. Mary’s school took in Elora students from Kindergarten through Grade 8.
In 2001, a renovated and expanded school was officially opened.
In 2008, a further expansion was built to meet the needs of the growing community. At this time, an additional six classrooms were added, along with new boys’ and girls’ washrooms, a handicapped washroom and two new offices.