本篇内容介绍了“PostgreSQL中StrategyGetBuffer函数有什么作用”的有关知识，在实际案例的操作过程中，不少人都会遇到这样的困境，接下来就让小编带领大家学习一下如何处理这些情况吧！希望大家仔细阅读，能够学有所成！
一、数据结构
BufferDesc
共享缓冲区的共享描述符(状态)数据
//buffer header锁定
#define BM_LOCKED (1U << 22)
//数据需要写入(标记为DIRTY)
#define BM_DIRTY (1U << 23)
//数据是有效的
#define BM_VALID (1U << 24)
//已分配buffer tag
#define BM_TAG_VALID (1U << 25)
//正在R/W
#define BM_IO_IN_PROGRESS (1U << 26)
//上一个I/O出现错误
#define BM_IO_ERROR (1U << 27)
//开始写则变DIRTY
#define BM_JUST_DIRTIED (1U << 28)
//存在等待sole pin的其他进程
#define BM_PIN_COUNT_WAITER (1U << 29)
//checkpoint发生,必须刷到磁盘上
#define BM_CHECKPOINT_NEEDED (1U << 30)
//持久化buffer(不是unlogged或者初始化fork)
#define BM_PERMANENT (1U << 31)
typedef struct BufferDesc
{
//buffer tag
BufferTag tag;
//buffer索引编号(0开始)
int buf_id;
//tag状态,包括flags/refcount和usagecount
pg_atomic_uint32 state;
//pin-count等待进程ID
int wait_backend_pid;
//空闲链表链中下一个空闲的buffer
int freeNext;
//缓冲区内容锁
LWLock content_lock;
} BufferDesc;BufferTag
Buffer tag标记了buffer存储的是磁盘中哪个block
typedef struct buftag
{
//物理relation标识符
RelFileNode rnode;
ForkNumber forkNum;
//相对于relation起始的块号
BlockNumber blockNum;
} BufferTag;SMgrRelation
smgr.c维护一个包含SMgrRelation对象的hash表,SMgrRelation对象本质上是缓存的文件句柄.
typedef struct SMgrRelationData
{
//-------- rnode是哈希表的搜索键,因此在结构体的首位
//关系物理定义ID
RelFileNodeBackend smgr_rnode;
//--------- 指向拥有的指针,如无则为NULL
struct SMgrRelationData **smgr_owner;
//当前插入的目标bloc
BlockNumber smgr_targblock;
//最后已知的fsm fork大小
BlockNumber smgr_fsm_nblocks;
//最后已知的vm fork大小
BlockNumber smgr_vm_nblocks;
//------- 未来可能新增的公共域
//存储管理器选择器
int smgr_which;
int md_num_open_segs[MAX_FORKNUM + 1];
struct _MdfdVec *md_seg_fds[MAX_FORKNUM + 1];
//如没有宿主,未宿主的SMgrRelations链表的链表链接.
struct SMgrRelationData *next_unowned_reln;
} SMgrRelationData;
typedef SMgrRelationData *SMgrRelation;RelFileNodeBackend
组合relfilenode和后台进程ID,用于提供需要定位物理存储的所有信息.
typedef struct RelFileNodeBackend
{
RelFileNode node;//节点
BackendId backend;//后台进程
} RelFileNodeBackend;StrategyControl
共享的空闲链表控制信息
typedef struct
{
//自旋锁,用于保护下面的值
slock_t buffer_strategy_lock;
pg_atomic_uint32 nextVictimBuffer;
//未使用的buffers链表头部
int firstFreeBuffer;
//未使用的buffers链表尾部
int lastFreeBuffer;
//完成一轮clock sweep循环,进行计数
uint32 completePasses;
//自上次重置后分配的buffers
pg_atomic_uint32 numBufferAllocs;
int bgwprocno;
} BufferStrategyControl;
//指向BufferStrategyControl结构体的指针
static BufferStrategyControl *StrategyControl = NULL;二、源码解读
StrategyGetBuffer在BufferAlloc()中,由bufmgr调用,用于获得下一个候选的buffer.
其主要的处理逻辑如下:
1.初始化相关变量
2.如策略对象不为空,则从环形缓冲区中获取buffer,如成功则返回buf
3.如需要,则唤醒后台进程bgwriter,从共享内存中读取一次,然后根据该值设置latch
4.计算buffer分配请求,这样bgwriter可以估算buffer消耗的比例.
5.检查freelist中是否存在buffer
5.1如存在,则执行相关判断逻辑,如成功,则返回buf
5.2如不存在
5.2.1则使用clock sweep算法,选择buffer,执行相关判断,如成功,则返回buf
5.2.2如无法获取,在尝试过trycounter次后,报错
BufferDesc *
StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state)
{
BufferDesc *buf;//buffer描述符
int bgwprocno;
int trycounter;//尝试次数
//避免重复的依赖和解依赖
uint32 local_buf_state;
if (strategy != NULL)
{
//从环形缓冲区中获取buffer,如获取成功,则返回该buffer
buf = GetBufferFromRing(strategy, buf_state);
if (buf != NULL)
return buf;
}
bgwprocno = INT_ACCESS_ONCE(StrategyControl->bgwprocno);
if (bgwprocno != -1)
{
//--- 如bgwprocno不是-1
//在设置latch前,首先重置bgwprocno为-1
StrategyControl->bgwprocno = -1;
SetLatch(&ProcGlobal->allProcs[bgwprocno].procLatch);
}
pg_atomic_fetch_add_u32(&StrategyControl->numBufferAllocs, 1);
if (StrategyControl->firstFreeBuffer >= 0)
{
while (true)
{
//请求自旋锁,删除空闲链表中的元素
SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
if (StrategyControl->firstFreeBuffer < 0)
{
//如无空闲空间,则马上跳出循环
SpinLockRelease(&StrategyControl->buffer_strategy_lock);
break;
}
//获取缓冲描述符
buf = GetBufferDescriptor(StrategyControl->firstFreeBuffer);
Assert(buf->freeNext != FREENEXT_NOT_IN_LIST);
//无条件的清除空闲链表中的buffer
StrategyControl->firstFreeBuffer = buf->freeNext;
buf->freeNext = FREENEXT_NOT_IN_LIST;
SpinLockRelease(&StrategyControl->buffer_strategy_lock);
//锁定缓冲头部
local_buf_state = LockBufHdr(buf);
if (BUF_STATE_GET_REFCOUNT(local_buf_state) == 0
&& BUF_STATE_GET_USAGECOUNT(local_buf_state) == 0)
{
//refcount == 0 && usagecount == 0
if (strategy != NULL)
//非默认策略,则添加到环形缓冲区中
AddBufferToRing(strategy, buf);
//设置输出参数
*buf_state = local_buf_state;
//返回buf
return buf;
}
//不满足条件,解锁buffer header
UnlockBufHdr(buf, local_buf_state);
}
}
//空闲链表中找不到或者满足不了条件,则执行"clock sweep"算法
//int NBuffers = 1000;
trycounter = NBuffers;//尝试次数
for (;;)
{
//------- 循环
//获取buffer描述符
buf = GetBufferDescriptor(ClockSweepTick());
local_buf_state = LockBufHdr(buf);
if (BUF_STATE_GET_REFCOUNT(local_buf_state) == 0)
{
//----- refcount == 0
if (BUF_STATE_GET_USAGECOUNT(local_buf_state) != 0)
{
//usage_count <> 0
//usage_count - 1
local_buf_state -= BUF_USAGECOUNT_ONE;
//重置尝试次数
trycounter = NBuffers;
}
else
{
//usage_count = 0
//发现一个可用的buffer
if (strategy != NULL)
//添加到该策略的环形缓冲区中
AddBufferToRing(strategy, buf);
//输出参数赋值
*buf_state = local_buf_state;
//返回buf
return buf;
}
}
else if (--trycounter == 0)
{
//----- refcount <> 0 && --trycounter == 0
UnlockBufHdr(buf, local_buf_state);
elog(ERROR, "no unpinned buffers available");
}
//解锁buffer header
UnlockBufHdr(buf, local_buf_state);
}
}三、跟踪分析
测试脚本,查询数据表:
10:01:54 (xdb@[local]:5432)testdb=# select * from t1 limit 10;启动gdb,设置断点
(gdb)
Continuing.
Breakpoint 1, StrategyGetBuffer (strategy=0x0, buf_state=0x7ffcc97fb4ec) at freelist.c:212
212 if (strategy != NULL)
(gdb)输入参数
strategy=NULL,策略对象,使用默认策略
(gdb) p *buf_state
$1 = 01.初始化相关变量
2.如策略对象不为空,则从环形缓冲区中获取buffer,如成功则返回buf
3.如需要,则唤醒后台进程bgwriter,从共享内存中读取一次,然后根据该值设置latch
(gdb) n
231 bgwprocno = INT_ACCESS_ONCE(StrategyControl->bgwprocno);
(gdb)
232 if (bgwprocno != -1)
(gdb)
235 StrategyControl->bgwprocno = -1;
(gdb) p bgwprocno
$2 = 112
(gdb) p StrategyControl
$3 = (BufferStrategyControl *) 0x7f8607b21700
(gdb) p *StrategyControl
$4 = {buffer_strategy_lock = 0 '\000', nextVictimBuffer = {value = 0}, firstFreeBuffer = 134, lastFreeBuffer = 65535,
completePasses = 0, numBufferAllocs = {value = 0}, bgwprocno = 112}
(gdb) n
242 SetLatch(&ProcGlobal->allProcs[bgwprocno].procLatch);
(gdb)4.计算buffer分配请求,这样bgwriter可以估算buffer消耗的比例.
(gdb)
250 pg_atomic_fetch_add_u32(&StrategyControl->numBufferAllocs, 1);5.检查freelist中是否存在buffer
(gdb)
268 if (StrategyControl->firstFreeBuffer >= 0)5.1如存在,则执行相关判断逻辑,如成功,则返回buf
(gdb) n
273 SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
(gdb)
275 if (StrategyControl->firstFreeBuffer < 0)
(gdb)
281 buf = GetBufferDescriptor(StrategyControl->firstFreeBuffer);
(gdb)
282 Assert(buf->freeNext != FREENEXT_NOT_IN_LIST);
(gdb) p *buf
$5 = {tag = {rnode = {spcNode = 0, dbNode = 0, relNode = 0}, forkNum = InvalidForkNumber, blockNum = 4294967295},
buf_id = 134, state = {value = 0}, wait_backend_pid = 0, freeNext = 135, content_lock = {tranche = 54, state = {
value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}}
(gdb) n
285 StrategyControl->firstFreeBuffer = buf->freeNext;
(gdb)
286 buf->freeNext = FREENEXT_NOT_IN_LIST;
(gdb)
292 SpinLockRelease(&StrategyControl->buffer_strategy_lock);
(gdb)
301 local_buf_state = LockBufHdr(buf);
(gdb)
302 if (BUF_STATE_GET_REFCOUNT(local_buf_state) == 0
(gdb)
303 && BUF_STATE_GET_USAGECOUNT(local_buf_state) == 0)
(gdb)
305 if (strategy != NULL)
(gdb)
307 *buf_state = local_buf_state;
(gdb)
308 return buf;
(gdb) p *buf_state
$6 = 4194304
(gdb) p *buf
$7 = {tag = {rnode = {spcNode = 0, dbNode = 0, relNode = 0}, forkNum = InvalidForkNumber, blockNum = 4294967295},
buf_id = 134, state = {value = 4194304}, wait_backend_pid = 0, freeNext = -2, content_lock = {tranche = 54, state = {
value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}}
(gdb)返回结果,回到BufferAlloc
(gdb) n
358 }
(gdb)
BufferAlloc (smgr=0x22a38a0, relpersistence=112 'p', forkNum=MAIN_FORKNUM, blockNum=0, strategy=0x0,
foundPtr=0x7ffcc97fb5c3) at bufmgr.c:1073
1073 Assert(BUF_STATE_GET_REFCOUNT(buf_state) == 0);
(gdb)“PostgreSQL中StrategyGetBuffer函数有什么作用”的内容就介绍到这里了，感谢大家的阅读。如果想了解更多行业相关的知识可以关注亿速云网站，小编将为大家输出更多高质量的实用文章！
