As first established by Edward III after 1348, the Royal College of St George comprised three principal architectural elements: a chapel, the Dean's Cloister and the Canons' Cloister. These core elements still constitute the heart of the collegiate complex and all continue in active use. In particular the two cloisters and the buildings around them have been the home of the Dean and canons for more than six hundred and fifty years, gradually evolving to reflect the changing needs of the community.

To understand the present layout of the chapel and cloisters it is necessary to look back beyond the 1348 foundation of the college by Edward III. The earliest remains in this area actually belong to a set of royal lodgings completed by Henry III in the 1240s. Attached to Henry III's lodgings (which roughly occupied the area of Canon's Cloister) was a small, enclosed area of lawn and a chapel dedicated to Edward the Confessor. And attached to them was an earlier 12th-century great hall, now entirely demolished. This lay in the area of what is now a green to the north of the present St George's Chapel.

These buildings materially conditioned the new College which Edward III began to build in 1348. The lawn attached to Henry III's lodgings was remodelled as the Dean's Cloister; the 12th-century great hall was passed over to the college community for their daily use and, most importantly of all, the existing chapel was remodelled and rededicated as the Chapel of the Blessed Virgin Mary, St George and St Edward the Confessor.

But the buildings completed by Edward III college have in turn undergone much alteration. The original chapel was rebuilt as a Lady Chapel by Henry VII after 1493 and then recast in its present form by Queen Victoria as a memorial to Prince Albert. The domestic buildings have also been extensively refashioned over time.