The mezuzah should go on the right door going from the garden as your son suggested. Even though the garden gate is rarely used, it represents another pathway into the garden from the street and renders all the outer doors of the house as “front doors,” in terms of mezuzah placement.[1]
The fact that the “back” door swings into the garden is not significant in this case as the “indication from the hinge” is only considered in the absence of other factors.[2]
Incidentally, if your garden gate has a lintel, it would need its own mezuzah even though it is infrequently used.[3]
[1] Agur B’ohalecha 28:3.
[2] Chayei Adam 15:8; Kitzur Shulchan Aruch 11:4; Cf. Agur B’ohalecha (27:14), who maintains that most authorities hold that the indication from the hinge takes precedent over the main room.
[3] Chovas HaDar 2:7.