Postgresql – the best way to compress postgres database dump file

CAUTION The answer about changing the UNIX password for "postgres" through "$ sudo passwd postgres" is not preferred, and can even be DANGEROUS!

This is why: By default, the UNIX account "postgres" is locked, which means it cannot be logged in using a password. If you use "sudo passwd postgres", the account is immediately unlocked. Worse, if you set the password to something weak, like "postgres", then you are exposed to a great security danger. For example, there are a number of bots out there trying the username/password combo "postgres/postgres" to log into your UNIX system.

What you should do is follow Chris James's answer:

sudo -u postgres psql postgres

# \password postgres

Enter new password: 

To explain it a little bit. There are usually two default ways to login to PostgreSQL server:

1. By running the "psql" command as a UNIX user (so-called IDENT/PEER authentication), e.g.: sudo -u postgres psql. Note that sudo -u does NOT unlock the UNIX user.

2. by TCP/IP connection using PostgreSQL's own managed username/password (so-called TCP authentication) (i.e., NOT the UNIX password).

So you never want to set the password for UNIX account "postgres". Leave it locked as it is by default.

Of course things can change if you configure it differently from the default setting. For example, one could sync the PostgreSQL password with UNIX password and only allow local logins. That would be beyond the scope of this question.

patch could be you friend could try creating a new dump, we'll call it current_dump and do a diff against old_dump then use

   diff old_dump new_dump > patch.input
    (copy the new_dump) to another file at this point new.db.dump)
    patch new.db.dump patch.input

You might have to sort the sort the dump files before you do the diff and depending on how big your database is this might not really be practical.

Also be very careful and do this against a backup until you are sure its right.

Otherwise, maybe there are ETL type tools that can do this.