"PostgreSQL PANIC: could not locate a valid checkpoint record"

I am not a DBA; however, throughout my career, I have often been tasked with deploying several databases for production. I have experience deploying MySQL, PostgreSQL, MSSQL, and Oracle (with FRA). In production, I adhere to all the best practices that I can find. However, in some QA situations, unexpected issues can arise. Recently, I encountered a situation where a disk filled up, and I had to clear the WAL archives from a Linux PostgreSQL server. While there are scripts and other correct methods to handle this, because it was a QA environment and I had snapshots, and also because it was not a busy DB, I chose to simply delete all the WAL archives and reboot. Until this time, this approach had worked every time. However, not this time...

The Postgres service would not start, After some investigation of the logs /mnt/logs/PostgreSQLXX/data/logs/postgresql-XX.log I saw:

PostgreSQL PANIC: could not locate a valid checkpoint record is an error that occurs when the database cannot find a valid checkpoint record in the transaction log.

This issue can be resolved by resetting the write-ahead log and other control information of the PostgreSQL database cluster.

Here are the steps I used to resolve:

1. Stop the postgresql server if it’s still running. This will be dependent on the backing Linux flavor however in my case its systemctl stop postgresql-xx <-Version
2. Switch to user postgres.

sudo su - postgres

3. Go to the path where pg_resetwal is present. On Ubuntu for PostgreSQL version 12, the path is:

cd /usr/lib/postgresql/12/bin

4. Run the following command to reset the write-ahead log:

./pg_resetwal -f /var/lib/postgresql/12/main

4. Restart the database server, you could just restart the service but after I have done something like this I like a fresh start :-)

This should allow you to recover from the PANIC error and start your PostgreSQL server again. Note that resetting the write-ahead log will result in data loss, so ensure you have a backup of your important data before performing this operation, in my case I knew there would be no issue as its a pretty idle db.

Anyway.. Enjoy that accessible data !