I need to programmatically insert 10's of millions of records into a postgres database. Presently I am executing 1000's of insert statements in a single "query".
Is there a better way to do this, some bulk insert statement I dont know about?
Postgresql – What’s the fastest way to do a bulk insert into Postgres
bulkinsertpostgresql
Related Topic
- Postgresql – Postgres: INSERT if does not exist already
- Mysql – How tonsert many rows into a MySQL table and return the new IDs
- Postgresql – Run a PostgreSQL .sql file using command line arguments
- Postgresql – PSQLException: current transaction is aborted, commands ignored until end of transaction block
- Postgresql – How to create User/Database in script for Docker Postgres
- Postgresql – How to persist data in a dockerized postgres database using volumes
- Postgresql – How to do a bulk insert with node-postgres
Best Answer
PostgreSQL has a guide on how to best populate a database initially, and they suggest using the COPY command for bulk loading rows. The guide has some other good tips on how to speed up the process, like removing indexes and foreign keys before loading the data (and adding them back afterwards).