The explanation is that Vegemite's characteristics (high yeast content, viscosity and strong smell) can be abused to break other jail prohibitions.

The yeast could be used to brew alcohol, and the viscosity and strong smell could be used to hide drugs and misdirect sniffer dogs.