The Horseshoe Cloister with its forest of tall chimneys is a splendid example of grand medieval timber-frame architecture. It was built between 1478 and 1482 as part of Edward IV's ambitious remodelling of the college and makes extensive use of brick, a material then in high fashion. The cloister was intended to provide accommodation for the so-called vicars of the college, junior clerics who helped in its daily round of divine service. There were originally 21 lodgings within it, each comprising a ground-floor living room and a bedroom above. Running round the inside face of the lodgings is a covered timber walk with ornate openings. This walk originally extended to the chapel and allowed the vicars to walk to and fro without being exposed to the weather.

Since its construction the cloister has undergone several radical changes of appearance. It was recast in its present form by Sir George Gilbert Scott in 1870-1, who tried to rescue the original medieval design of the buildings. The colourful copper roofs are an addition of the 1950s.

In the area around the Horseshoe Cloister in the Middle Ages were several subsidiary buildings connected with the service of St George's Chapel including accommodation and a schoolroom for the boys in the choir. Several of these buildings have been demolished but the remainder, which extend along the north wall of the castle, as well as the cloister itself, continue to house members of the college community including the organist and lay clerks.