I found the following line in the output:
[Sun Sep 15 15:13:43 CEST 2019] Sign failed: "detail":"Error creating new cert :: too many certificates already issued for exact set of domains: xxxxxxx.ca,xxxxxxx.de,xxxxxxx.es,xxxxxxx.fr,xxxxxxx.it,xxxxxxx.net,xxxxxxx.nl,xxxxxxx.us,cron.xxxxxxx.net,webhook.xxxxxxx.net,www.xxxxxxx.ca,www.xxxxxxx.de,www.xxxxxxx.es,www.xxxxxxx.fr,www.xxxxxxx.it,www.xxxxxxx.net,www.xxxxxxx.nl,www.xxxxxxx.us: see https://letsencrypt.org/docs/rate-limits/"
As a result, I seem to get a faulty cert from LE, instead of no cert at all. Then, when restarting Apache, it fails with "Configuration failed".