Haa, this is something I've done some research on!
The most change you'll get will be almost certainly temporary. Actual penile atrophy is a VERY long term thing. Now, I guess if you really are wearing break then "temporary" is "as long as you're still wearing to where a full erection cannot be sustained."
But yeah, common scientific consensus is that actual long-term shrinkage is only caused by aging and rarely long-term medication. HRT, for example, is documented to cause testes shrinkage but actual penile atrophy is somewhat spurious as a connection; it can make it near impossible to get fully hard, but if medication is halted then size shouldn't be lost.