Pikeun ngaoptimalkeun patarosan PostgreSQL, kamampuan pikeun nganalisis sajarah kagiatan, khususna, ngantosan, konci, sareng statistik tabel, diperyogikeun pisan.
ekstensi pgsentinel :
«Sadaya inpormasi akumulasi disimpen ukur dina RAM, sareng jumlah mémori anu dikonsumsi diatur ku jumlah rékaman anu terakhir disimpen.
Widang queryid ditambahkeun - queryid sarua tina extension pg_stat_statements (pre-instalasi diperlukeun).«
Ieu, tangtosna, bakal ngabantosan pisan, tapi anu paling nyusahkeun nyaéta titik anu munggaran.Sadaya inpormasi akumulasi disimpen ukur dina RAM ”, nyaéta. aya dampak dina dasar target. Salaku tambahan, teu aya sajarah konci sareng statistik méja. Jelema. solusi umumna nyarios teu lengkep: "Henteu aya pakét anu siap-siap pikeun dipasang. Disarankeun pikeun ngaunduh sumber sareng ngumpul perpustakaan sorangan. Anjeun mimitina kudu masang pakét "devel" pikeun server Anjeun tur nyetel jalur ka pg_config dina variabel PATH.".
Sacara umum, aya loba fuss, sarta dina kasus database produksi serius, eta bisa jadi teu mungkin ngalakukeun nanaon jeung server. Urang kudu datang nepi ka hal urang sorangan deui.
Perhatosan.
Kusabab volume anu rada ageung sareng kusabab periode tés anu teu lengkep, tulisan éta utamina pikeun tujuan inpormasi, tinimbang sakumpulan tesis sareng hasil panengah.
Bahan anu langkung rinci bakal disiapkeun engké, dina bagian-bagian
Draf syarat pikeun solusi
Perlu ngembangkeun alat anu ngamungkinkeun anjeun pikeun nyimpen:
pg_stat_activity tempoan sajarah Riwayat konci sési nganggo tampilan pg_locks
Sarat solusi- ngaleutikan dampak dina database target.
Pamanggih umum- agén pangumpulan data diluncurkeun sanés dina pangkalan data target, tapi dina pangkalan data ngawaskeun salaku jasa systemd. Sumuhun, sababaraha leungitna data mungkin, tapi ieu teu kritis pikeun ngalaporkeun, tapi euweuh dampak dina database target dina watesan memori sareng spasi disk. Sareng upami nganggo kolam renang sambungan, dampak kana prosés pangguna minimal.
Tahap palaksanaan
1.Service tabel
Skéma anu misah dianggo pikeun nyimpen tabel, supados henteu ngahesekeun analisis tabel utama anu dianggo.
DROP SCHEMA IF EXISTS activity_hist ;
CREATE SCHEMA activity_hist AUTHORIZATION monitor ;
Nu penting: Skéma teu dijieun dina database target, tapi dina database monitoring.
pg_stat_activity tempoan sajarah
Méja dipaké pikeun nyimpen snapshots tina pintonan pg_stat_activity
activity_hist.history_pg_stat_activity :
--ACTIVITY_HIST.HISTORY_PG_STAT_ACTIVITY
DROP TABLE IF EXISTS activity_hist.history_pg_stat_activity;
CREATE TABLE activity_hist.history_pg_stat_activity
(
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text ,
queryid bigint
);
Pikeun nyepetkeun sisipan - euweuh indéks atawa larangan.
Pikeun nyimpen sajarah sorangan, tabel partitioned dipaké:
activity_hist.archive_pg_stat_activity :
DROP TABLE IF EXISTS activity_hist.archive_pg_stat_activity;
CREATE TABLE activity_hist.archive_pg_stat_activity
(
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text ,
queryid bigint
)
PARTITION BY RANGE (timepoint);
Kusabab dina hal ieu euweuh sarat pikeun speed sisipan, sababaraha indexes geus dijieun pikeun nyepetkeun kreasi laporan.
Sajarah blocking sési
Méja dipaké pikeun nyimpen snapshots tina konci sési ayeuna:
activity_hist.history_locking :
--ACTIVITY_HIST.HISTORY_LOCKING
DROP TABLE IF EXISTS activity_hist.history_locking;
CREATE TABLE activity_hist.history_locking
(
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
);
Ogé, pikeun nyepetkeun sisipan, teu aya indéks atanapi larangan.
Pikeun nyimpen sajarah sorangan, tabel partitioned dipaké:
activity_hist.archive_locking:
DROP TABLE IF EXISTS activity_hist.archive_locking;
CREATE TABLE activity_hist.archive_locking
(
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
)
PARTITION BY RANGE (timepoint);
Kusabab dina hal ieu euweuh sarat pikeun speed sisipan, sababaraha indexes geus dijieun pikeun nyepetkeun kreasi laporan.
2.Ngeusi sajarah ayeuna
Pikeun langsung ngumpulkeun snapshots view, skrip bash digunakeun anu ngajalankeun pungsi plpgsql.
plpgsql Fungsi dblink ngaksés pintonan dina pangkalan data target sareng nyelapkeun barisan kana tabel jasa dina pangkalan data ngawaskeun.
get_current_activity.sql
CREATE OR REPLACE FUNCTION activity_hist.get_current_activity( current_host text , current_s_name text , current_s_pass text ) RETURNS BOOLEAN AS $$
DECLARE
database_rec record;
dblink_str text ;
BEGIN
EXECUTE 'SELECT dblink_connect(''LINK1'',''host='||current_host||' port=5432 dbname=postgres'||
' user='||current_s_name||' password='||current_s_pass|| ' '')';
--------------------------------------------------------------------
--GET pg_stat_activity stats
INSERT INTO activity_hist.history_pg_stat_activity
(
SELECT * FROM dblink('LINK1',
'SELECT
now() ,
datid ,
datname ,
pid ,
usesysid ,
usename ,
application_name ,
client_addr ,
client_hostname ,
client_port ,
backend_start ,
xact_start ,
query_start ,
state_change ,
wait_event_type ,
wait_event ,
state ,
backend_xid ,
backend_xmin ,
query ,
backend_type
FROM pg_stat_activity
')
AS t (
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text
)
);
---------------------------------------
--ACTIVITY_HIST.HISTORY_LOCKING
INSERT INTO activity_hist.history_locking
(
SELECT * FROM dblink('LINK1',
'SELECT
now() ,
lock.locktype,
lock.relation,
lock.mode,
lock.transactionid as tid,
lock.virtualtransaction as vtid,
lock.pid,
pg_blocking_pids(lock.pid),
lock.granted
FROM pg_catalog.pg_locks lock LEFT JOIN pg_catalog.pg_database db ON db.oid = lock.database
WHERE NOT lock.pid = pg_backend_pid()
')
AS t (
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
)
);
PERFORM dblink_disconnect('LINK1');
RETURN TRUE ;
END
$$ LANGUAGE plpgsql;
Pikeun ngumpulkeun snapshots view, jasa systemd sareng dua skrip dianggo:
pg_current_activity.service
# /etc/systemd/system/pg_current_activity.service
[Unit]
Description=Collect history of pg_stat_activity , pg_locks
Wants=pg_current_activity.timer
[Service]
Type=forking
StartLimitIntervalSec=0
ExecStart=/home/postgres/pgutils/demon/get_current_activity.sh 10.124.70.40 postgres postgres
[Install]
WantedBy=multi-user.target
pg_current_activity.timer
# /etc/systemd/system/pg_current_activity.timer
[Unit]
Description=Run pg_current_activity.sh every 1 second
Requires=pg_current_activity.service
[Timer]
Unit=pg_current_activity.service
OnCalendar=*:*:0/1
AccuracySec=1
[Install]
WantedBy=timers.target
Pasihan hak kana naskah:
# chmod 755 pg_current_activity.timer
# chmod 755 pg_current_activity.service
Ku kituna, sajarah pintonan dikumpulkeun dina bentuk snapshots detik-demi-detik. Tangtosna, upami sadayana ditinggalkeun sapertos kitu, méja-méja bakal gancang-gancang ningkat ukuranana sareng langkung-langkung kirang produktif bakal janten teu mungkin.
Ieu diperlukeun pikeun ngatur arsip data.
3. Arsip sajarah
Pikeun arsip, arsip tabel partisi * dianggo.
partisi anyar dijieun unggal jam, bari data heubeul dipiceun tina sajarah * tabel, jadi ukuran sajarah * tabel teu robah teuing jeung speed sisipan teu nguraikeun kana waktu.
Nyiptakeun bagian anyar dilakukeun ku fungsi plpgsql activity_hist.archive_current_activity. Algoritma gawé basajan pisan (ngagunakeun conto bagian pikeun tabel archive_pg_stat_activity).
Jieun tur eusian bagian anyar
EXECUTE format(
'CREATE TABLE ' || partition_name ||
' PARTITION OF activity_hist.archive_pg_stat_activity FOR VALUES FROM ( %L ) TO ( %L ) ' ,
to_char(date_trunc('year', partition_min_range ),'YYYY')||'-'||
to_char(date_trunc('month', partition_min_range ),'MM')||'-'||
to_char(date_trunc('day', partition_min_range ),'DD')||' '||
to_char(date_trunc('hour', partition_min_range ),'HH24')||':00',
to_char(date_trunc('year', partition_max_range ),'YYYY')||'-'||
to_char(date_trunc('month', partition_max_range ),'MM')||'-'||
to_char(date_trunc('day', partition_max_range ),'DD')||' '||
to_char(date_trunc('hour', partition_max_range ),'HH24')||':00'
);
INSERT INTO activity_hist.archive_pg_stat_activity
(
SELECT *
FROM activity_hist.history_pg_stat_activity
WHERE timepoint BETWEEN partition_min_range AND partition_max_range
);
Nyiptakeun indéks
EXECUTE format (
'CREATE INDEX '||index_name||
' ON '||partition_name||' ( wait_event_type , backend_type , timepoint )'
);
EXECUTE format ('CREATE INDEX '||index_name||
' ON '||partition_name||' ( wait_event_type , backend_type , timepoint , queryid )'
);
Nyoplokkeun data heubeul tina tabel history_pg_stat_activity
DELETE
FROM activity_hist.history_pg_stat_activity
WHERE timepoint < partition_max_range;
Tangtosna, ti waktos ka waktos, bagian-bagian lami dihapus salaku teu perlu.
laporan dasar
Sabenerna, naha sadayana ieu dilakukeun? Pikeun ménta laporan pisan vaguely reminiscent tina Oracle urang AWR.
Penting pikeun nambihan yén pikeun nampi laporan, anjeun kedah ngawangun sambungan antara pg_stat_activity sareng pg_stat_statements view. Tabél dikaitkeun ku nambahkeun kolom 'queryid' kana tabel 'history_pg_stat_activity', 'archive_pg_stat_activity'. Métode pikeun nambahkeun nilai kolom téh saluareun ruang lingkup artikel ieu sarta dijelaskeun di dieu − pg_stat_statements + pg_stat_activity + loq_query = pg_ash? .
TOTAL WAKTU CPU pikeun QUERIES
Kahoyong :
WITH hist AS
(
SELECT
aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND backend_type = 'client backend' AND datname != 'postgres' AND ( aa.wait_event_type IS NULL ) ANDaa.state = 'active'
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND backend_type = 'client backend' AND datname != 'postgres' AND ( ha.wait_event_type IS NULL )AND ha.state = 'active'
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
WITH hist AS
(
SELECT
aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( aa.wait_event_type IS NOT NULL )
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( ha.wait_event_type IS NOT NULL )
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
WITH hist AS
(
SELECT
aa.wait_event_type , aa.wait_event
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
aa.wait_event IS NOT NULL
GROUP BY aa.wait_event_type , aa.wait_event
UNION
SELECT
ha.wait_event_type , ha.wait_event
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
ha.wait_event IS NOT NULL
GROUP BY ha.wait_event_type , ha.wait_event
)
SELECT wait_event_type , wait_event
FROM hist
GROUP BY wait_event_type , wait_event
ORDER BY 1 ASC,2 ASC
----------------------------------------------------------------------
WITH hist AS
(
SELECT
aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( aa.wait_event_type = waitings_stat_rec.wait_event_type AND aa.wait_event = waitings_stat_rec.wait_event )
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( ha.wait_event_type = waitings_stat_rec.wait_event_type AND ha.wait_event = waitings_stat_rec.wait_event )
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
SELECT
MIN(date_trunc('second',timepoint)) AS started ,
count(*) * interval '1 second' as duration ,
pid , blocking_pids , relation , mode , locktype
FROM
activity_hist.archive_locking al
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY pid , blocking_pids , relation , mode , locktype
UNION
SELECT
MIN(date_trunc('second',timepoint)) AS started ,
count(*) * interval '1 second' as duration ,
pid , blocking_pids , relation , mode , locktype
FROM
activity_hist.history_locking
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY pid , blocking_pids , relation , mode , locktype
ORDER BY 1
SELECT
blocking_pids
FROM
activity_hist.archive_locking al
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY blocking_pids
UNION
SELECT
blocking_pids
FROM
activity_hist.history_locking
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY blocking_pids
ORDER BY 1
---------------------------------------------------------------
SELECT
pid , usename , application_name , datname ,
MIN(date_trunc('second',timepoint)) as started ,
count(*) * interval '1 second' as duration ,
state ,
query
FROM activity_hist.archive_pg_stat_activity
WHERE pid= current_pid AND
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour')
GROUP BY pid , usename , application_name ,
datname ,
state_change,
state ,
query
UNION
SELECT
pid , usename , application_name , datname ,
MIN(date_trunc('second',timepoint)) as started ,
count(*) * interval '1 second' as duration ,
state ,
query
FROM activity_hist.history_pg_stat_activity_for_reports
WHERE pid= current_pid AND
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour')
GROUP BY pid , usename , application_name ,
datname ,
state_change,
state ,
query
ORDER BY 5 , 1
Patarosan dasar anu ditampilkeun sareng laporan anu hasilna parantos ngajantenkeun kahirupan langkung gampang nalika nganalisa insiden kinerja.
Dumasar kana patarosan dasar, anjeun tiasa nampi laporan anu samar-samar nyarupaan AWR Oracle. Conto laporan ringkesan
+------------------------------------------------- ------------------------------------ | LAPORAN KONSOLIDASI Pikeun Kagiatan jeung Nungguan.
Ngalajengkeun. Salajengna dina garis nyaéta nyiptakeun sajarah konci (pg_stat_locks), pedaran anu langkung rinci ngeunaan prosés ngeusian méja.