Infertile men have an increased risk of developing high grade prostate cancer, which is more likely to grow and spread quickly, a new study has found.