PostgreSQL
PostgreSQL是一個開源的,社區驅動的,符合標準的 對象-關係型 資料庫系統。
安裝[編輯 | 編輯原始碼]
安裝 postgresql包,這同時會創建一個名為 postgres 的新系統用戶。
你現在可以通過提權工具來切換到 postgres 用戶下。
[postgres]$
作為前置符號。你可以通過下列命令來切換到 PostgreSQL 用戶下:
初始化配置[編輯 | 編輯原始碼]
在可以正常使用 PostgreSQL 之前,資料庫集群必須被初始化:
[postgres]$ initdb -D /var/lib/postgres/data
其中 -D
提供了資料庫集群的默認數據存放位置(如果需要修改目錄位置,可以參考#修改默認數據目錄)。initdb
也支持多種其它的命令行參數:
- By default, the locale and the encoding for the database cluster are derived from your current environment (using $LANG value). If this is not what you want, you can override the defaults using
--locale=locale
(where locale is to be chosen amongst the system's available locales) and--encoding=encoding
(which must match the chosen locale). (Once the database is up, you can check which values were used with[postgres]$ psql -l
.) - 如果數據目錄所在文件系統沒有數據校驗和功能,你可能會想要啟用 PostgreSQL 自帶的校驗和功能來提高數據完整性保障 - 使用
--data-checksums
參數即可。相關細節可參考 #Enable data checksumming。(Once the database is up, you can check if it is enabled with[postgres]$ psql -c "SHOW data_checksums"
.) - The trust authentication method is used by default, meaning that anyone on the host can connect as any database user. You can use
--auth-local=peer --auth-host=scram-sha-256
for safer authentication methods. - For more options, see
initdb --help
and official documentation.
Example:
[postgres]$ initdb --locale=C.UTF-8 --encoding=UTF8 -D /var/lib/postgres/data --data-checksums
Many lines should now appear on the screen with several ending by ... ok
:
The files belonging to this database system will be owned by user "postgres". This user must also own the server process. The database cluster will be initialized with locale "C.UTF-8". The default text search configuration will be set to "english". Data page checksums are enabled. creating directory /var/lib/postgres/data ... ok creating subdirectories ... ok selecting dynamic shared memory implementation ... posix selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting default time zone ... UTC creating configuration files ... ok running bootstrap script ... ok performing post-bootstrap initialization ... ok syncing data to disk ... ok initdb: warning: enabling "trust" authentication for local connections initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. Success. You can now start the database server using: pg_ctl -D /var/lib/postgres/data -l logfile start
If these are the kind of lines you see, then the process succeeded. Return to the regular user using exit
.
WARNING
, see #Restricts access rights to the database superuser by default./var/lib/postgres
, you will have to edit the service file. If the root is under home
, make sure to set ProtectHome
to false.- 如果資料庫位於Btrfs文件系統上,你應該在創建資料庫前禁用資料庫目錄的Copy-on-Write
- 如果資料庫位於 ZFS 文件系統上,你應該在創建資料庫前先查閱 ZFS#Databases。
最後,啟動並啟用 postgresql.service
服務。
創建第一個資料庫/用戶[編輯 | 編輯原始碼]
以 postgres 用戶身份, 使用 createuser 命令添加一個新的資料庫用戶:
[postgres]$ createuser --interactive
使用 createdb 命令,創建一個上述用戶可以讀寫的新資料庫(execute this command from your login shell if the database user has the same name as your Linux user, otherwise add -O database-username
to the following command):
$ createdb myDatabaseName
-U postgres
to the previous command.熟悉 PostgreSQL[編輯 | 編輯原始碼]
連接資料庫 shell[編輯 | 編輯原始碼]
登錄為 postgres 用戶,啟動主資料庫 shell psql,你可以創建/刪除資料庫或表、配置權限和運行 SQL 命令。使用 -d
選項連接你創建的資料庫(如果沒有指定資料庫,psql
會嘗試連接與你用戶名同名的資料庫)。
[postgres]$ psql -d myDatabaseName
一些有用的命令:
- 獲取幫助
=> \help
- 列出所有資料庫
=> \l
- 連接到特定資料庫
=> \c database
- 列出所有用戶以及他們的權限
=> \du
- 展示當前資料庫中所有的表相關的匯總信息
=> \dt
- 退出
psql
=> \q 或是 Ctrl+d
當然也有更多元命令,但這些應該能夠幫助您開始。要查看所有元命令:
=> \?
可選配置[編輯 | 編輯原始碼]
The PostgreSQL database server configuration file is postgresql.conf
. This file is located in the data directory of the server, typically /var/lib/postgres/data
. This folder also houses the other main configuration files, including the pg_hba.conf
which defines authentication settings, for both local users and other hosts ones.
find
and locate
are not finding the configuration files.Restricts access rights to the database superuser by default[編輯 | 編輯原始碼]
The defaults pg_hba.conf
allow any local user to connect as any database user, including the database superuser.
This is likely not what you want, so in order to restrict global access to the postgres user, change the following line:
/var/lib/postgres/data/pg_hba.conf
# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all all trust
To:
/var/lib/postgres/data/pg_hba.conf
# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all postgres peer
You might later add additional lines depending on your needs or software ones.
Require password for login[編輯 | 編輯原始碼]
Edit /var/lib/postgres/data/pg_hba.conf
and set the authentication method for each user (or "all" to affect all users) to scram-sha-256
(preferred), or md5
(less secure; should be avoided if possible):
/var/lib/postgres/data/pg_hba.conf
# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all user scram-sha-256
If you choose scram-sha-256
, you must also edit /var/lib/postgres/data/postgresql.conf
and set:
/var/lib/postgres/data/postgresql.conf
password_encryption = scram-sha-256
Restart postgresql.service
, and then re-add each user's password using ALTER USER user WITH ENCRYPTED PASSWORD 'password';
.
Configure PostgreSQL to be accessible exclusively through UNIX Sockets[編輯 | 編輯原始碼]
In the connections and authentications section of your configuration, set:
/var/lib/postgres/data/postgresql.conf
listen_addresses = ''
This will disable network listening completely.
After this you should restart postgresql.service
for the changes to take effect.
Configure PostgreSQL to be accessible from remote hosts[編輯 | 編輯原始碼]
In the connections and authentications section, set the listen_addresses
line to your needs:
/var/lib/postgres/data/postgresql.conf
listen_addresses = 'localhost,my_local_ip_address'
You can use '*'
to listen on all available addresses.
5432
by default for remote connections. Make sure this port is open in your firewall and able to receive incoming connections. You can also change it in the configuration file, right below listen_addresses
Then add a line like the following to the authentication config:
/var/lib/postgres/data/pg_hba.conf
# TYPE DATABASE USER ADDRESS METHOD # IPv4 local connections: host all all ip_address/32 md5
where ip_address
is the IP address of the remote client.
See the documentation for pg_hba.conf.
After this you should restart postgresql.service
for the changes to take effect.
For troubleshooting take a look in the server log file:
# journalctl -u postgresql.service
Configure PostgreSQL authenticate against PAM[編輯 | 編輯原始碼]
PostgreSQL offers a number of authentication methods. If you would like to allow users to authenticate with their system password, additional steps are necessary. First you need to enable PAM for the connection.
For example, the same configuration as above, but with PAM enabled:
/var/lib/postgres/data/pg_hba.conf
# IPv4 local connections: host all all my_remote_client_ip_address/32 pam
The PostgreSQL server is however running without root privileges and will not be able to access /etc/shadow
. We can work around that by allowing the postgres group to access this file:
# setfacl -m g:postgres:r /etc/shadow
修改默認數據目錄[編輯 | 編輯原始碼]
默認設置下,新創建的資料庫會被存放於 /var/lib/postgres/data
目錄下。如果要更改目錄位置,可以參考下列步驟:
創建一個新文件夾,並將其所有者設為 postgres 用戶:
# mkdir -p /pathto/pgroot/data # chown -R postgres:postgres /pathto/pgroot
切換到 postgres 用戶,然後初始化新集群:
[postgres]$ initdb -D /pathto/pgroot/data
通過編輯 postgresql.service
來附加配置片段,以覆蓋 Environment
和 PIDFile
設置。例如:
/etc/systemd/system/postgresql.service.d/PGROOT.conf
[Service] Environment=PGROOT=/pathto/pgroot PIDFile=/pathto/pgroot/data/postmaster.pid
如果你想使用 /home
目錄作為默認目錄或用於表空間,需要在此文件中額外添加一行:
ProtectHome=false
Change default encoding of new databases to UTF-8[編輯 | 編輯原始碼]
initdb
with -E UTF8
or while using an UTF-8 locale, these steps are not required.When creating a new database (e.g. with createdb blog
) PostgreSQL actually copies a template database. There are two predefined templates: template0
is vanilla, while template1
is meant as an on-site template changeable by the administrator and is used by default. In order to change the encoding of a new database, one of the options is to change on-site template1
. To do this, log into PostgreSQL shell (psql
) and execute the following:
First, we need to drop template1
. Templates cannot be dropped, so we first modify it so it is an ordinary database:
UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';
Now we can drop it:
DROP DATABASE template1;
The next step is to create a new database from template0
, with a new default encoding:
CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UNICODE';
Now modify template1
so it is actually a template:
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1';
Optionally, if you do not want anyone connecting to this template, set datallowconn
to FALSE
:
UPDATE pg_database SET datallowconn = FALSE WHERE datname = 'template1';
pg_upgrade
.Now you can create a new database:
[postgres]$ createdb blog
If you log back in to psql
and check the databases, you should see the proper encoding of your new database:
\l
List of databases Name | Owner | Encoding | Collation | Ctype | Access privileges -----------+----------+-----------+-----------+-------+---------------------- blog | postgres | UTF8 | C | C | postgres | postgres | SQL_ASCII | C | C | template0 | postgres | SQL_ASCII | C | C | =c/postgres : postgres=CTc/postgres template1 | postgres | UTF8 | C | C |
Enable data checksumming[編輯 | 編輯原始碼]
If your database files reside on a file system without checksumming, its data is suspectible to silent data corruption due to bit rot and broken hardware. While those events are rare, you might want to enable PostgreSQL's built-in data checksumming if you care about data integrity. This feature must be enabled on the cluster level, not per-database or per-table.
- There is a minimal performance impact, especially while reading large datasets from disk. In-memory operations are not affected.
- PostgreSQL is unable to repair corrupt data - it will only abort transactions reading from corrupt pages to prevent further damage or invalid execution results.
- Checksums cover on-disk data (row) pages only, not metadata or control structures. In-memory pages are not checksummed. Error-corrected storage and ECC memory is still beneficial.
- To enable checksumming during cluster creation, add the
--data-checksums
argument toinitdb
. - To verify whenever checksumming is enabled, run
[postgres]$ psql -c "SHOW data_checksums"
(which should printoff
oron
). - To toggle checksumming on an existing cluster:
- Stop
postgresql.service
. - Run
[postgres]$ pg_checksums --pgdata /var/lib/postgres/data --enable
(or--disable
if you no longer want checksumming). Enabling checksums will rewrite all database pages, which will take a while for large database instances. - Start
postgresql.service
.
圖形化管理工具[編輯 | 編輯原始碼]
- phpPgAdmin — Web-based administration tool for PostgreSQL.
- pgAdmin — Comprehensive design and management GUI for PostgreSQL.
- https://www.pgadmin.org/ || pgadmin3AUR or pgadmin4包[損壞的連結:package not found]
- pgModeler — Graphical schema designer for PostgreSQL.
- Postbird — Cross-platform PostgreSQL GUI client, written in JavaScript, runs with Electron.
For tools supporting multiple DBMSs, see List of applications/Documents#Database tools.
升級 PostgreSQL[編輯 | 編輯原始碼]
Upgrading major PostgreSQL versions (e.g. version 14.x to version 15.y) requires some extra maintenance.
Get the currently used database version via
# cat /var/lib/postgres/data/PG_VERSION
To ensure you do not accidentally upgrade the database to an incompatible version, it is recommended to skip updates to the PostgreSQL packages.
Minor version upgrades are safe to perform. However, if you do an accidental upgrade to a different major version, you might not be able to access any of your data. Always check the PostgreSQL home page to be sure of what steps are required for each upgrade. For a bit about why this is the case, see the versioning policy.
There are two main ways to upgrade your PostgreSQL database. Read the official documentation for details.
pg_upgrade[編輯 | 編輯原始碼]
The pg_upgrade
utility attempts to copy over as much compatible data as possible between clusters and upgrading everything else. It is generally the fastest method to upgrade most instances, although it requires access to binaries for both source and target PostgreSQL versions. Read the pg_upgrade(1) man page to understand what actions it performs. For non-trivial instances (e.g. with streaming replication or log-shipping), read the upstream documentation first.
For those wishing to use pg_upgrade
, a postgresql-old-upgrade包 package is available that will always run one major version behind the real PostgreSQL package. This can be installed side-by-side with the new version of PostgreSQL. To upgrade from older versions of PostgreSQL there are AUR packages available, e.g. postgresql-12-upgradeAUR. (You must use the pg_upgrade
version packaged with the PostgreSQL version you are upgrading to.)
Note that the database cluster directory does not change from version to version, so before running pg_upgrade
, it is necessary to rename your existing data directory and migrate into a new directory. The new database cluster must be initialized using the same parameters as the old one.
When you are ready to begin the upgrade:
- While the old database cluster is still online, collect the
initdb
arguments used to create it. Refer to #Initial configuration for more information. - Stop
postgresql.service
. (Check the unit status to be sure that PostgresSQL was stopped correctly. If it failed,pg_upgrade
will fail too.) - Upgrade postgresql包, postgresql-libs包, and postgresql-old-upgrade包.
- Rename the old cluster directory, then create a new cluster and temporary working directory:
# mv /var/lib/postgres/data /var/lib/postgres/olddata # mkdir /var/lib/postgres/data /var/lib/postgres/tmp # chown postgres:postgres /var/lib/postgres/data /var/lib/postgres/tmp [postgres]$ cd /var/lib/postgres/tmp
- Initialize the new cluster using the same
initdb
arguments as were used for the old cluster:[postgres]$ initdb -D /var/lib/postgres/data --locale=C.UTF-8 --encoding=UTF8 --data-checksums
- Upgrade the cluster, replacing
PG_VERSION
below, with the old PostgreSQL version number (e.g.14
):[postgres]$ pg_upgrade -b /opt/pgsql-PG_VERSION/bin -B /usr/bin -d /var/lib/postgres/olddata -D /var/lib/postgres/data
注意:- If necessary, adjust the configuration files of new cluster (e.g.
pg_hba.conf
andpostgresql.conf
) to match the old cluster. - If
pg_upgrade
fails withThe source cluster was not shut down cleanly
, PostgreSQL was not stopped before attempting the upgrade. Stop it, then restart the cluster with old database binaries to recover old cluster files:[postgres]$ /opt/pgsql-PG_VERSION/bin/pg_ctl start -D /var/lib/postgres/olddata && /opt/pgsql-PG_VERSION/bin/pg_ctl stop -D /var/lib/postgres/olddata
You can now safely retry thepg_upgrade
command. If the upgrade process still fails, shut down any database processes, downgrade PostgreSQL packages to their previous versions, recover your previous cluster data from backups and restart the upgrade process.
- If necessary, adjust the configuration files of new cluster (e.g.
- Start
postgresql.service
again. - Optional: Run
[postgres]$ vacuumdb --all --analyze-in-stages
to recalculate query analyzer statistics, which should improve query performance shortly after the upgrade. (Adding--jobs=NUMBER_OF_CPU_CORES
argument may improve this command's performance.) - Optional: Back up the
/var/lib/postgres/olddata
directory in case you need to restore a previous PostgreSQL version. - Delete the
/var/lib/postgres/olddata
directory with old cluster data. - Delete the
/var/lib/postgres/tmp
directory.
Caveats[編輯 | 編輯原始碼]
pg_upgrade
is said to be tricky. On a talk in a recent postgresql conference, the presenter asked everyone that had used pg_upgrade
to raise their hands, followed up by the next question - how many have used pg_upgrade
and it just worked on the first attempt without any errors? Very few hands to be seen on the second question. This section is intended to be a list of problems people have observed when upgrading PostgreSQL under Archlinux (problems should preferably be reported upstream, too).
- When upgrading from Postgresql 14 to Postgresql 15, old postgresql did not start - it insisted on fetching data from
/var/lib/postgres/data
rather than/var/lib/postgres/data.old
and threw an error message that the directory was created with pg 15 and not pg 14. Moving the old directory to/var/lib/postgres/data
and the new directory to/var/lib/postgres/15
did not help - then things were apparently freezing. What did help was to edit/var/lib/postgres/data.old/postgres.conf
and removedata_directory
from that file. (Hm ... perhaps a user error to have that thing there at all, if the config file is in the data directory, obviously postgresql has already found the data directory and should not need that directive. Postgresql was installed throughpuppetlabs-postgresql
puppet module, it is managing the configuration).
Manual dump and reload[編輯 | 編輯原始碼]
You could also do something like this (after the upgrade and install of postgresql-old-upgrade包).
- Below are the commands for upgrading from PostgreSQL 14. You can find similar commands in
/opt/
for your version of PostgreSQL cluster, provided you have matching version of postgresql-old-upgrade包 package installed. - If you had customized your
pg_hba.conf
file, you may have to temporarily modify it to allow full access to old database cluster from local system. After upgrade is complete set your customization to new database cluster as well and restartpostgresql.service
.
Stop postgresql.service
# mv /var/lib/postgres/data /var/lib/postgres/olddata # mkdir /var/lib/postgres/data # chown postgres:postgres /var/lib/postgres/data [postgres]$ initdb -D /var/lib/postgres/data --locale=C.UTF-8 --encoding=UTF8 --data-checksums [postgres]$ /opt/pgsql-14/bin/pg_ctl -D /var/lib/postgres/olddata/ start # cp /usr/lib/postgresql/postgis-3.so /opt/pgsql-14/lib/ # Only if postgis installed [postgres]$ pg_dumpall -h /tmp -f /tmp/old_backup.sql [postgres]$ /opt/pgsql-14/bin/pg_ctl -D /var/lib/postgres/olddata/ stop
Start postgresql.service
[postgres]$ psql -f /tmp/old_backup.sql postgres
Troubleshooting[編輯 | 編輯原始碼]
Improve performance of small transactions[編輯 | 編輯原始碼]
If you are using PostgresSQL on a local machine for development and it seems slow, you could try turning synchronous_commit off in the configuration. Beware of the caveats, however.
/var/lib/postgres/data/postgresql.conf
synchronous_commit = off
空閒時防止磁碟寫入[編輯 | 編輯原始碼]
PostgreSQL periodically updates its internal "statistics" file. By default, this file is stored on disk, which prevents disks from spinning down on laptops and causes hard drive seek noise. It is simple and safe to relocate this file to a memory-only file system with the following configuration option:
/var/lib/postgres/data/postgresql.conf
stats_temp_directory = '/run/postgresql'
pgAdmin 4 issues after upgrade to PostgreSQL 12[編輯 | 編輯原始碼]
If you see errors about string indices must be integers
when navigating the tree on the left, or about column rel.relhasoids does not exist
when viewing the data, remove the server from the connection list in pgAdmin and add a fresh server instance. pgAdmin will otherwise continue to treat the server as a PostgreSQL 11 server resulting in these issues.
PostgreSQL database unable to start after package update when using extensions[編輯 | 編輯原始碼]
The cause in this case is mostly the existing package is not compiled for the newer version (and it may be up-to-date), the solution is rebuilding the package either manually or waiting for an update to the extension package.
Failing to start a PostgreSQL server with the older version of the database while upgrading to the newer version with extensions[編輯 | 編輯原始碼]
This is caused because the old version of postgres from the package postgresql-old-upgrade包 does not have the required extensions (.so files) in its lib directory, the current solution is dirty, and might cause a lot of problems so keep a backup of the database just in case, basically copy the required extension .so files from /usr/lib/postgresql/
to /opt/pgsql-XX/lib/
(remember to replace XX with the major version of postgresql-old-upgrade包).
For example, for timescaledb
# cp /usr/lib/postgresql/timescaledb*.so /opt/pgsql-13/lib/
/opt/pgsql-XX/
to know the exact files to copy, check the contant of the package of the extension using :
$ pacman -Ql package_name