Measuring PostgreSQL Checkpoint Statistics

内容导读

互联网集市收集整理的这篇技术教程文章主要介绍了Measuring PostgreSQL Checkpoint Statistics，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含7572字，纯文字阅读大概需要11分钟。

内容图文

　　The problem with checkpoints is that they can be very intensive, because to complete one requires writing every single bit of changed data in the database’s buffer cache out to disk. There were a number of features added to PostgreSQL 8.3 that allow you to better monitor the checkpoint overhead, and to lower it by spreading the activity over a longer period of time. I wrote a long article about those changes called Checkpoints and the Background Writer that goes over what changed, but it’s pretty dry reading.
　　

　　What you probably want to know is how to monitor checkpoints on your production system, and how to tell if they’re happening too often. Even though things have improved, “checkpoint spikes” where disk I/O becomes really heavy are still possible even in current PostgreSQL versions. And it doesn’t help that the default configuration is tuned for very low disk space and fast crash recovery rather than performance. The checkpoint_segments parameter that’s one input on how often a checkpoint happens defaults to 3, which forces a checkpoint after only 48MB of writes.
　　

　　You can find out checkpoint frequency two ways. You can turn on log_checkpoints and watch what happens in the logs. You can also use the pg_stat_bgwriter view, which gives a count of each of the two sources for checkpoints (time passing and writes occurring) as well as statistics about how much work they did.
　　

　　The main problem with making that easier to do is that until recently, it’s been impossible to reset the counters inside of pg_stat_bgwriter. That means you have to take a snapshot with a timestamp on it, wait a while, take another snapshot, then subtract all the values to derive any useful statistics from the data. That’s a pain.

　　Enough of a pain that I wrote a patch to make it easier. With the current development version of the database, you can now call pg_stat_reset_shared(‘bgwriter’) and pop all these values back to 0 again. This allows following a practice that used to be common on PostgreSQL. Before 8.3, there was a parameter named stats_reset_on_server_start you could turn on. That reset all of the server’s internal statistics each time you started it. That meant that you could call the handy pg_postmaster_start_time() function, compare with the current time, and always have an accurate count in terms of operations/second of any statistic available on the system.
It’s still not automatic, but now that resetting these shared pieces is possible you can do it yourself. The first key is to integrate statistics clearing into your server startup sequence. A script like this will work:

pg_ctl start -l $PGLOG -w
psql -c "select pg_stat_reset();"
psql -c "select pg_stat_reset_shared(‘bgwriter‘);"

　　Note the “-w” on the start command there–that will make pg_ctl wait until the server is finished starting before it returns, which is vital if you want to immediately execute a statement against it.
If you’ve done that, and your server start time is essentially the same as when the background writer stats started collection, you can now use this fun query:

SELECT
total_checkpoints,
seconds_since_start / total_checkpoints / 60 AS minutes_between_checkpoints
FROM
(SELECT
EXTRACT(EPOCH FROM (now() - pg_postmaster_start_time())) AS seconds_since_start
(checkpoints_timed+checkpoints_req) AS total_checkpoints
FROM pg_stat_bgwriter
) AS sub;

And get a simple report of exactly how often checkpoints are happening on your system. The output looks like this:

total_checkpoints           | 9
minutes_between_checkpoints | 3.82999310740741

　　What you do with this information is stare at the average time interval and see if it seems too fast. Normally, you’d want a checkpoint to happen no more than every five minutes, and on a busy system you might need to push it to ten minutes or more to have a hope of keeping up. With this example, every 3.8 minutes is probably too fast–this is a system that needs checkpoint_segments to be higher.
　　Using this technique to measure the checkpoint interval lets you know if you need to increase the checkpoint_segments and checkpoint_timeout parameters in order to achieve that goal. You can compute the numbers manually right now, and once 9.0 ships it’s something you can consider making completely automatic–so long as you don’t mind your stats going away each time the server restarts.
　　There are some other interesting ways to analyze the data the background writer provides for you in pg_stat_bgwriter, but I’m not going to give away all of my tricks today.

注：

1、上面给了一个查询checkpoint执行时间长度的sql，当然在计算之前要清掉历史记录。直接运行select pg_stat_reset()是清不掉的，需要执行select pg_stat_reset_shared(‘bgwriter‘)，这可以将视图pg_stat_bgwriter中的各值除掉stats_reset均置为0；

2、视图pg_stat_bgwriter 字段：

swrd=# \d pg_stat_bgwriter 
              View "pg_catalog.pg_stat_bgwriter"
        Column         |           Type           | Modifiers 
-----------------------+--------------------------+-----------
 checkpoints_timed     | bigint                   | 
 checkpoints_req       | bigint                   | 
 checkpoint_write_time | double precision         | 
 checkpoint_sync_time  | double precision         | 
 buffers_checkpoint    | bigint                   | 
 buffers_clean         | bigint                   | 
 maxwritten_clean      | bigint                   | 
 buffers_backend       | bigint                   | 
 buffers_backend_fsync | bigint                   | 
 buffers_alloc         | bigint                   | 
 stats_reset           | timestamp with time zone | 

其中checkpoints_timed表示由于checkpoint_timeout 引起的checkpoint的次数，checkpoints_req表示由于checkpoint_segments引起的checkpoint的次数。手动执行checkpoint命令，会将次数计算到checkpoints_req字段中，根据这两个的大小情况，可以来决定修改checkpoint_timeout 和checkpoint_segments值的大小。

两字段数值相加就是总的checkpoint数，可以结合buffers_checkpoint值计算出平均每次checkpoint的buffer大小。

3、计算checkpoint时间的sql：

SELECT
total_checkpoints,
seconds_since_start / total_checkpoints / 60 AS minutes_between_checkpoints
FROM
(SELECT
EXTRACT(EPOCH FROM (now() - pg_postmaster_start_time())) AS seconds_since_start
(checkpoints_timed+checkpoints_req) AS total_checkpoints
FROM pg_stat_bgwriter
) AS sub;

EPOCH：

The  Unix epoch   (or  Unix time   or  POSIX time   or  Unix timestamp ) is the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT), not counting leap seconds (in ISO 8601: 1970-01-01T00:00:00Z). Literally speaking the epoch is Unix time 0 (midnight 1-1-1970), but ‘epoch‘ is often used as a synonym for ‘Unix time‘. Many Unix systems store epoch dates as a signed 32-bit integer, which might cause problems on January 19, 2038 (known as the Year 2038 problem or Y2038).  

EXTRACT：

EXTRACT(field FROM source)

The extract function retrieves subfields such as year or hour from date/time values. source must be a value expression of type timestamp, time, or interval. (Expressions of type date are cast to timestamp and can therefore be used as well.) field is an identifier or string that selects what field to extract from the source value. The extract function returns values of type double precision.

参考：

http://blog.2ndquadrant.com/measuring_postgresql_checkpoin/

http://www.westnet.com/~gsmith/content/postgresql/chkp-bgw-83.htm

http://yao.iteye.com/blog/628941

http://www.postgresql.org/docs/9.4/static/functions-datetime.html

Measuring PostgreSQL Checkpoint Statistics

标签：

本文系统来源：http://www.cnblogs.com/xiaotengyi/p/5024311.html

内容总结

以上是互联网集市为您收集整理的Measuring PostgreSQL Checkpoint Statistics全部内容，希望文章能够帮你解决Measuring PostgreSQL Checkpoint Statistics所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。

内容备注

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。

内容手机端