Hi Norm,
You could alter your intro to;
"I know what you were thinking, an 80 year-old with an allotment".
You could go on to say that you DO have an allotment, primarily because the 6 x 4 potting shed gives you your own space. The potting shed could then become your hypothetical "den of iniquity" with references during your routine.
I enjoy stand-ups more when they use an underlying theme, particularly if the parting punch references it.