背景
最关键的一些数据库健康指标,趋势监测。
1 总连接数
主要看趋势,直接与业务量挂钩
如果连接数接近max_connection水位,需要注意。
同时连接数应与数据库主机可用内存挂钩,每个连接保守估计10MB内存开销(这里还未计算SYSCACHE,RELCACHE)。
select count(*) from pg_stat_activity ;
演示,打印每秒的总连接数。
psql select count(*) from pg_stat_activity ; \watch 1
2 N秒内新建的连接数
主要看趋势,直接与业务量挂钩
如果突发大量连接,可能是新增了业务服务器,或者是性能抖动过导致业务大量新建连接满足并发的请求。
突然连接数下降,可能原因是业务服务器突然释放连接,或者业务服务器挂了。
select count(*) from pg_stat_activity where now()-backend_start > '? second';
演示,打印每秒的5秒内新建连接数。
psql select count(*) from pg_stat_activity where now()-backend_start > '5 second'; \watch 1
3 SQL活跃统计
需要加载pg_stat_statements,如果需要跟踪IO时间,需要开启track_io_timing。
同时需要注意,由于pg_stat_statements跟踪的SQL有限,最近未访问过的SQL的跟踪信息可能被抛弃。所以统计并不是非常的精准。
postgres=# \d pg_stat_statements View "public.pg_stat_statements" Column | Type | Collation | Nullable | Default ---------------------+------------------+-----------+----------+--------- userid | oid | | | dbid | oid | | | queryid | bigint | | | query | text | | | calls | bigint | | | total_time | double precision | | | min_time | double precision | | | max_time | double precision | | | mean_time | double precision | | | stddev_time | double precision | | | rows | bigint | | | shared_blks_hit | bigint | | | shared_blks_read | bigint | | | shared_blks_dirtied | bigint | | | shared_blks_written | bigint | | | local_blks_hit | bigint | | | local_blks_read | bigint | | | local_blks_dirtied | bigint | | | local_blks_written | bigint | | | temp_blks_read | bigint | | | temp_blks_written | bigint | | | blk_read_time | double precision | | | blk_write_time | double precision | | |
3.1 QPS
QPS指标来自pg_stat_statements,由于这个插件有一个STATEMENT采集上限,可配置,例如最多采集1000条SQL,如果有新的SQL被采集到时,并且1000已用完,则会踢掉最老的SQL。所以我们这里统计的QPS并不是完全精确,不过还好PG内部会自动合并SQL,把一些条件替换成变量,这样即使不使用绑定变量,也能追踪到很多SQL。
对于业务SQL非常繁多并且大多数都是活跃SQL的场景,可以适当调大pg_stat_statements的track数,提高精准度。
除此之外,可以改进pg_stat_statements的功能,直接统计精准的QPS。
主要看趋势,直接与业务量挂钩
with a as (select sum(calls) s, sum(case when ltrim(query,' ') ~* '^select' then calls else 0 end) q from pg_stat_statements), b as (select sum(calls) s, sum(case when ltrim(query,' ') ~* '^select' then calls else 0 end) q from pg_stat_statements , pg_sleep(1)) select b.s-a.s, -- QPS b.q-a.q, -- 读QPS b.s-b.q-a.s+a.q -- 写QPS from a,b;
如果只想看QPS,使用
with a as (select sum(calls) s from pg_stat_statements), b as (select sum(calls) s from pg_stat_statements , pg_sleep(1)) select b.s-a.s -- QPS from a,b;
演示,打印每秒的QPS。
psql with a as (select sum(calls) s from pg_stat_statements), b as (select sum(calls) s from pg_stat_statements , pg_sleep(1)) select b.s-a.s -- QPS from a,b; \watch 0.000001
3.2 每秒处理了多少行
每秒处理了多少行,包括写入,读取,更新,删除等操作。
两次快照相减除以时间间隔sum(pg_stat_statements.rows)
3.3 共享缓冲区:每秒缓存命中、未命中读
shared_blks_hit | bigint | | | shared_blks_read | bigint | | |
3.4 共享缓冲区:每秒产生多少脏页
shared_blks_dirtied | bigint | | |
3.5 共享缓冲区:每秒异步write多少脏页
shared_blks_written | bigint | | |
3.6 进程本地缓冲区:每秒缓存命中、未命中读
local_blks_hit | bigint | | | local_blks_read | bigint | | |
3.7 进程本地缓冲区:每秒产生多少脏页
local_blks_dirtied | bigint | | |
3.8 进程本地缓冲区:每秒异步write多少脏页
local_blks_written | bigint | | |
3.9 临时文件每秒读
temp_blks_read | bigint | | |
3.10 临时文件每秒写
temp_blks_written | bigint | | |
3.11 两次快照之间的读数据块耗时
blk_read_time | double precision | | |
3.12 两次快照之间的写数据块耗时
blk_write_time | double precision | | |
4 active session
主要看趋势,直接与业务量挂钩
如果活跃会话数长时间超过CPU核数时,说明数据库响应变慢了,需要深刻关注。
select count(*) from pg_stat_activity where state='active';
演示,打印每秒的活跃会话数。
psql select count(*) from pg_stat_activity where state='active'; \watch 1
5 平均RT
活跃会话/qps = RT(秒)
6 long query
当前系统中执行时间超过N秒的SQL有多少条,LONG QUERY与活跃会话的比例说明当前LONG SQL的占比。占比越高,说明该系统可能偏向OLAP,占比越低,说明该系统偏向OLTP业务。
select count(*) from pg_stat_activity where state='active' and now()-query_start > interval '? second';
演示,打印每秒系统中执行时间超过5秒的SQL有多少条。
psql select count(*) from pg_stat_activity where state='active' and now()-query_start > interval '5 second'; \watch 1
7 long transaction
当前系统中N秒未结束的事务有多少条
select count(*) from pg_stat_activity where now()-xact_start > interval '? second';
演示,打印每秒系统中5秒未结束的事务有多少条
psql select count(*) from pg_stat_activity where now()-xact_start > interval '5 second'; \watch 1
8 idle in transaction
当前系统中在事务中并且处于空闲状态的会话有多少,很多,说明业务端的处理可能比较慢,如果结合锁等待发现有大量锁等待,并且活跃会话数有突增,可能需要关注并排查业务逻辑的问题。
select count(*) from pg_stat_activity where state='idle in transaction';
演示,打印每秒系统中在事务中并且处于空闲状态的会话有多少
psql select count(*) from pg_stat_activity where state='idle in transaction'; \watch 1
9 long idle in transaction
当前系统中,有多少长期(超过N秒)处于空闲的事务。如果有较多这样的事务,说明业务端的处理时间超过N秒的情况非常普遍,应该尽快排查业务。
比如前端开启了游标,等待用户的翻页动作,用户可能开小差了。又比如业务上使用了一些交互模式,等用户的一些输入等。
这种情况应该尽量避免,否则长时间占用连接资源。
select count(*) from pg_stat_activity where state='idle in transaction' and now()-state_change > interval '? second';
演示,打印每秒系统中在事务中并且处于空闲状态(超过5秒)的会话有多少
psql select count(*) from pg_stat_activity where state='idle in transaction' and now()-state_change > interval '5 second'; \watch 1
10 waiting
当前系统中,处于等待中的会话有多少。
如果很多,说明出现了大量的锁等待,使用末尾文章进行排查。
select count(*) from pg_stat_activity where wait_event_type is not null;
演示,打印每秒系统中处于等待中的会话有多少。
psql select count(*) from pg_stat_activity where wait_event_type is not null; \watch 1
11 long waiting
当前系统中,等待超过N秒的会话有多少。
select count(*) from pg_stat_activity where wait_event_type is not null and now()-state_change > interval '? second';
演示,打印每秒系统中等待超过5秒的会话有多少。
psql select count(*) from pg_stat_activity where wait_event_type is not null and now()-state_change > interval '5 second'; \watch 1
12 2pc
当前系统中,2PC的事务有多少。如果接近max_prepared_transactions,需要注意。建议调大max_prepared_transactions,或者排查业务是否未及时提交。
select count(*) from pg_prepared_xacts;
演示,打印每秒系统中未结束的2PC事务数。
psql select count(*) from pg_prepared_xacts; \watch 1
13 long 2pc
当前系统中,超过N秒未结束的2PC的事务有多少。如果很多,需要排查业务为什么未及时提交。
select count(*) from pg_prepared_xacts where now() - prepared > interval '? second';
演示,打印每秒系统中5秒仍未结束的2PC事务数。
psql select count(*) from pg_prepared_xacts where now() - prepared > interval '5 second'; \watch 1
14 膨胀点监测 - 多久以前的垃圾可以被回收
时间间隔越大,说明越容易导致膨胀。
排查这几个方向,长事务,长SQL,2PC,持有SNAPSHOT的QUERY。必要时把不合理的老的会话干掉。
with a as (select min(xact_start) m from pg_stat_activity where backend_xid is not null or backend_xmin is not null), b as (select min(prepared) m from pg_prepared_xacts) select now()-least(a.m,b.m) from a,b;
演示,打印每秒系统中多久以前的垃圾可以被回收
psql with a as (select min(xact_start) m from pg_stat_activity where backend_xid is not null or backend_xmin is not null), b as (select min(prepared) m from pg_prepared_xacts) select now()-least(a.m,b.m) from a,b; \watch 1
15 空间
看当前占用情况,打快照,看时间维度空间变化情况。
按库划分
postgres=# \l+ List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace | Description -----------+----------+-----------+------------+------------+-----------------------+---------+------------+-------------------------------------------- postgres | postgres | SQL_ASCII | en_US.UTF8 | en_US.UTF8 | | 54 GB | pg_default | default administrative connection database template0 | postgres | SQL_ASCII | en_US.UTF8 | en_US.UTF8 | =c/postgres +| 7489 kB | pg_default | unmodifiable empty database | | | | | postgres=CTc/postgres | | | template1 | postgres | SQL_ASCII | en_US.UTF8 | en_US.UTF8 | =c/postgres +| 578 MB | pg_default | default template for new databases | | | | | postgres=CTc/postgres | | | test | test | SQL_ASCII | en_US.UTF8 | en_US.UTF8 | | 7489 kB | pg_default | (4 rows)
按表空间划分
postgres=# \db+ List of tablespaces Name | Owner | Location | Access privileges | Options | Size | Description --------------------+----------+--------------------------------------+-------------------+---------+---------+------------- dbt2_index1 | postgres | /data02/pg/tbs_tpcc/index1/ts | | | 452 MB | dbt2_index2 | postgres | /data02/pg/tbs_tpcc/index2/ts | | | 869 MB | dbt2_pk_customer | postgres | /data02/pg/tbs_tpcc/pk_customer/ts | | | 451 MB | dbt2_pk_district | postgres | /data02/pg/tbs_tpcc/pk_district/ts | | | 236 kB | dbt2_pk_item | postgres | /data02/pg/tbs_tpcc/pk_item/ts | | | 2212 kB | dbt2_pk_new_order | postgres | /data02/pg/tbs_tpcc/pk_new_order/ts | | | 149 MB | dbt2_pk_order_line | postgres | /data02/pg/tbs_tpcc/pk_order_line/ts | | | 4701 MB | dbt2_pk_orders | postgres | /data02/pg/tbs_tpcc/pk_orders/ts | | | 490 MB | dbt2_pk_stock | postgres | /data02/pg/tbs_tpcc/pk_stock/ts | | | 1768 MB | dbt2_pk_warehouse | postgres | /data02/pg/tbs_tpcc/pk_warehouse/ts | | | 44 kB | pg_default | postgres | | | | 46 GB | pg_global | postgres | | | | 573 kB | (12 rows)
16 数据空间
数据占用的空间。
17 日志空间
WAL日志占用的空间。
18 备库发送延迟
select application_name,client_addr,client_hostname,client_port,state,sync_priority,sync_state,pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), sent_lsn)) from pg_stat_replication;
19 备库APPLY延迟
select application_name,client_addr,client_hostname,client_port,state,sync_priority,sync_state,pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lag)) from pg_stat_replication;
20 SLOT 延迟
select slot_name, plugin, slot_type, temporary, active, active_pid, pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) from pg_replication_slots;
21 归档延迟
最后一次归档失败时间减去最后一次归档成功的时间,求时间差。
select last_failed_time - last_archived_time from pg_stat_archiver;
22 数据库活动信息
以下都可以针对单个数据库输出,也可以输出整个实例的统计。
postgres=# \d pg_stat_database View "pg_catalog.pg_stat_database" Column | Type | Collation | Nullable | Default ----------------+--------------------------+-----------+----------+--------- datid | oid | | | datname | name | | | numbackends | integer | | | xact_commit | bigint | | | xact_rollback | bigint | | | blks_read | bigint | | | blks_hit | bigint | | | tup_returned | bigint | | | tup_fetched | bigint | | | tup_inserted | bigint | | | tup_updated | bigint | | | tup_deleted | bigint | | | conflicts | bigint | | | temp_files | bigint | | | temp_bytes | bigint | | | deadlocks | bigint | | | blk_read_time | double precision | | | blk_write_time | double precision | | | stats_reset | timestamp with time zone | | |
22.1 每秒事务提交数
多次查询计算
select sum(xact_commit) from pg_stat_database; -- pg_stat_get_db_xact_commit 为stable函数,一个事务中两次调用之间只执行一次,所以需要外部多次执行。
22.2 每秒事务回滚数
select sum(xact_rollback) from pg_stat_database;
22.3 每秒全表扫描记录数
select sum(tup_returned) from pg_stat_database;
22.4 每秒索引扫描回表记录数
select sum(tup_fetched) from pg_stat_database;
22.5 每秒插入记录数
select sum(tup_inserted) from pg_stat_database;
22.6 每秒更新记录数
select sum(tup_updated) from pg_stat_database;
22.7 每秒删除记录数
select sum(tup_deleted) from pg_stat_database;
22.8 备库查询冲突数
select sum(conflicts) from pg_stat_database;
22.9 死锁数
select sum(deadlocks) from pg_stat_database;
来源地址:https://blog.csdn.net/liuyunshengsir/article/details/129949008