It appears that I can't use my mail server over SSL whilst using a Let's encrypt certificate, it says it's an invalid certificate. I can use this email in other email clients like the default windows mail client.