PostgreSQL – PostgreSQL Won’t Start Anymore: Troubleshooting Guide

Debian 9.5 – postgresql cluster 9.4. It's not a fresh installation – it was a perfectly decently running system, until I tried restarting the service

all commands: /etc/init.d/postgresql start and service postgresql start and systemctl start postgresql behave the same: the the prompt hangs for about 10-15 seconds, and then nothing happens – postgres isn't running!

here's the syslog:

Jul 18 18:15:35 intranet systemd[1]: Starting PostgreSQL Cluster 9.4-main...
Jul 18 18:16:06 intranet postgresql@9.4-main[27610]: The PostgreSQL server failed to start. Please check the log output.
Jul 18 18:16:06 intranet systemd[1]: postgresql@9.4-main.service: Control process exited, code=exited status=1
Jul 18 18:16:06 intranet systemd[1]: Failed to start PostgreSQL Cluster 9.4-main.
Jul 18 18:16:06 intranet systemd[1]: postgresql@9.4-main.service: Unit entered failed state.
Jul 18 18:16:06 intranet systemd[1]: postgresql@9.4-main.service: Failed with result 'exit-code'.

The postgres-log (/var/log/postgresql/) is empty.

journalctl -xe:

-- Unit postgresql@9.4-main.service has begun starting up.
Jul 18 18:16:06 intranet postgresql@9.4-main[27610]: The PostgreSQL server failed to start. Please check the log output.
Jul 18 18:16:06 intranet systemd[1]: postgresql@9.4-main.service: Control process exited, code=exited status=1
Jul 18 18:16:06 intranet systemd[1]: Failed to start PostgreSQL Cluster 9.4-main.
-- Subject: Unit postgresql@9.4-main.service has failed

systemctl status postgresql

● postgresql.service - PostgreSQL RDBMS
   Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
   Active: active (exited) since Wed 2018-07-18 17:42:02 CEST; 37min ago
 Main PID: 29934 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   CGroup: /system.slice/postgresql.service

Jul 18 17:42:02 intranet systemd[1]: Starting PostgreSQL RDBMS...
Jul 18 17:42:02 intranet systemd[1]: Started PostgreSQL RDBMS.

systemctl status postgresql@9.4-main

● postgresql@9.4-main.service - PostgreSQL Cluster 9.4-main
   Loaded: loaded (/lib/systemd/system/postgresql@.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2018-07-18 18:16:06 CEST; 1 day 14h ago
  Process: 27610 ExecStart=postgresql@9.4-main --skip-systemctl-redirect 9.4-main start (code=exited, status=1/FAILURE)

Jul 18 18:15:35 intranet systemd[1]: Starting PostgreSQL Cluster 9.4-main...
Jul 18 18:16:06 intranet postgresql@9.4-main[27610]: The PostgreSQL server failed to start. Please check the log output.
Jul 18 18:16:06 intranet systemd[1]: postgresql@9.4-main.service: Control process exited, code=exited status=1
Jul 18 18:16:06 intranet systemd[1]: Failed to start PostgreSQL Cluster 9.4-main.
Jul 18 18:16:06 intranet systemd[1]: postgresql@9.4-main.service: Unit entered failed state.
Jul 18 18:16:06 intranet systemd[1]: postgresql@9.4-main.service: Failed with result 'exit-code'.

So there are absolutely no indicators about what's wrong. Strangely, the system was running without any problems all along – my application just crashed, and I tried restarting postgres – and now it wont start again.

After several hours of searching, I discovered, that strangely, it can be launched manually.

I logged as the postgres-user and used the command:

/usr/lib/postgresql/9.4/bin/postgres -D /var/lib/postgresql/9.4/main -c config_file=/etc/postgresql/9.4/main/postgresql.conf

it just works fine – no errors, no thing. I even tried adding the '-d 3' to get more detailed output ….however there is nothing out of the ordinary. it runs fine – the problem are just the init-scripts. How do I fix this – or at least find out the root of the problem?