Andrana hamorona analogue ny ASH ho an'ny PostgreSQL
Fanambarana olana
Mba hanamafisana ny fangatahana PostgreSQL dia ilaina ny fahaizana mamakafaka ny tantaran'ny hetsika, indrindra ny fiandrasana, ny hidin-trano ary ny antontan'isa latabatra.
pgsentinel extension :
Β«Ny fampahalalana voaangona rehetra dia voatahiry ao amin'ny RAM ihany, ary ny habetsaky ny fitadidiana lany dia fehezin'ny isan'ny rakitra voatahiry farany.
Nampiana ny saha queryid - ilay queryid mitovy amin'ny fanitarana pg_stat_statements (takina mialoha ny fametrahana).Β«
Mazava ho azy fa hanampy betsaka izany, fa ny tena manahirana dia ny teboka voalohany. βNy fampahalalana voaangona rehetra dia voatahiry ao anaty RAM ihany β, i.e. misy fiantraikany amin'ny fototra kendrena. Ankoatr'izay, tsy misy ny tantaran'ny hidin-trano sy ny antontan'isa latabatra. Ireo. ny vahaolana amin'ny ankapobeny dia tsy feno: "Tsy mbola misy fonosana efa vita ho an'ny fametrahana. Soso-kevitra ny misintona ny loharano ary manangona ny tranomboky ny tenanao. Mila mametraka ny fonosana "devel" ho an'ny mpizara anao aloha ianao ary mametraka ny lalana mankany amin'ny pg_config ao amin'ny PATH variable.".
Amin'ny ankapobeny, be dia be ny tabataba, ary raha ny angon-drakitra famokarana matotra dia mety tsy ho azo atao ny manao na inona na inona amin'ny mpizara. Mila mamorona zavatra ho antsika manokana indray isika.
Warning.
Noho ny haben'ny habeny sy noho ny vanim-potoana fitsapana tsy feno, ny lahatsoratra dia natao indrindra ho an'ny tanjona fampahafantarana, fa ho toy ny andian-dahatsoratra sy valiny manelanelana.
Ny fitaovana amin'ny antsipiriany bebe kokoa dia homanina any aoriana, amin'ny ampahany
Drafitra takiana amin'ny vahaolana
Ilaina ny manamboatra fitaovana ahafahanao mitahiry:
pg_stat_activity mijery tantara Tantaran'ny fanakatonana fivoriana mampiasa ny fijery pg_locks
Fitakiana vahaolanaβ manamaivana ny fiantraikany amin'ny angon-drakitra kendrena.
Hevitra ankapobenyβ Tsy ao amin'ny angon-drakitra kendrena no atomboka ny mpandraharaha mpanangom-baovao, fa ao amin'ny angon-drakitra fanaraha-maso ho serivisy systemd. Eny, mety hisy ny fahaverezan'ny angon-drakitra sasany, saingy tsy dia zava-dehibe amin'ny tatitra izany, saingy tsy misy fiantraikany amin'ny angon-drakitra kendrena amin'ny resaka fitadidiana sy habaka kapila. Ary amin'ny fampiasana dobo fampifandraisana dia kely ny fiantraikany amin'ny fizotran'ny mpampiasa.
Dingana fampiharana
1. latabatra serivisy
Ny schema mitokana dia ampiasaina hitahirizana latabatra, mba tsy hanasarotra ny famakafakana ireo tabilao lehibe ampiasaina.
DROP SCHEMA IF EXISTS activity_hist ;
CREATE SCHEMA activity_hist AUTHORIZATION monitor ;
Zava-dehibe: Ny schema dia tsy noforonina ao amin'ny angon-drakitra kendrena, fa ao amin'ny angon-drakitra fanaraha-maso.
pg_stat_activity mijery tantara
Ny latabatra dia ampiasaina hitahiry ny sary amin'izao fotoana izao amin'ny fijery pg_stat_activity
activity_hist.history_pg_stat_activity :
--ACTIVITY_HIST.HISTORY_PG_STAT_ACTIVITY
DROP TABLE IF EXISTS activity_hist.history_pg_stat_activity;
CREATE TABLE activity_hist.history_pg_stat_activity
(
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text ,
queryid bigint
);
Mba hanafainganana ny fampidirana - tsy misy fanondroana na fameperana.
Mba hitehirizana ny tantara mihitsy, dia ampiasaina ny latabatra fisarahana:
activity_hist.archive_pg_stat_activity :
DROP TABLE IF EXISTS activity_hist.archive_pg_stat_activity;
CREATE TABLE activity_hist.archive_pg_stat_activity
(
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text ,
queryid bigint
)
PARTITION BY RANGE (timepoint);
Koa satria amin'ity tranga ity dia tsy misy fepetra takiana amin'ny hafainganam-pandehan'ny fampidirana, misy indexes noforonina mba hanafaingana ny famoronana tatitra.
Tantara fanakanana fivoriana
Ny latabatra dia ampiasaina hitehirizana ireo sary amin'izao fotoana izao momba ny hidin-trano:
activity_hist.history_locking :
--ACTIVITY_HIST.HISTORY_LOCKING
DROP TABLE IF EXISTS activity_hist.history_locking;
CREATE TABLE activity_hist.history_locking
(
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
);
Ary koa, mba hanafainganana ny fampidirana dia tsy misy index na fameperana.
Mba hitehirizana ny tantara mihitsy, dia ampiasaina ny latabatra fisarahana:
activity_hist.archive_locking:
DROP TABLE IF EXISTS activity_hist.archive_locking;
CREATE TABLE activity_hist.archive_locking
(
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
)
PARTITION BY RANGE (timepoint);
Koa satria amin'ity tranga ity dia tsy misy fepetra takiana amin'ny hafainganam-pandehan'ny fampidirana, misy indexes noforonina mba hanafaingana ny famoronana tatitra.
2. Famenoana ny tantara ankehitriny
Mba hanangonana mivantana snapshots dia ampiasaina ny script bash izay mampandeha ny asa plpgsql.
plpgsql Ny fiasa dblink dia miditra amin'ny fijery ao amin'ny angon-drakitra kendrena ary mampiditra andalana ao anaty tabilao serivisy ao amin'ny angon-drakitra fanaraha-maso.
get_current_activity.sql
CREATE OR REPLACE FUNCTION activity_hist.get_current_activity( current_host text , current_s_name text , current_s_pass text ) RETURNS BOOLEAN AS $$
DECLARE
database_rec record;
dblink_str text ;
BEGIN
EXECUTE 'SELECT dblink_connect(''LINK1'',''host='||current_host||' port=5432 dbname=postgres'||
' user='||current_s_name||' password='||current_s_pass|| ' '')';
--------------------------------------------------------------------
--GET pg_stat_activity stats
INSERT INTO activity_hist.history_pg_stat_activity
(
SELECT * FROM dblink('LINK1',
'SELECT
now() ,
datid ,
datname ,
pid ,
usesysid ,
usename ,
application_name ,
client_addr ,
client_hostname ,
client_port ,
backend_start ,
xact_start ,
query_start ,
state_change ,
wait_event_type ,
wait_event ,
state ,
backend_xid ,
backend_xmin ,
query ,
backend_type
FROM pg_stat_activity
')
AS t (
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text
)
);
---------------------------------------
--ACTIVITY_HIST.HISTORY_LOCKING
INSERT INTO activity_hist.history_locking
(
SELECT * FROM dblink('LINK1',
'SELECT
now() ,
lock.locktype,
lock.relation,
lock.mode,
lock.transactionid as tid,
lock.virtualtransaction as vtid,
lock.pid,
pg_blocking_pids(lock.pid),
lock.granted
FROM pg_catalog.pg_locks lock LEFT JOIN pg_catalog.pg_database db ON db.oid = lock.database
WHERE NOT lock.pid = pg_backend_pid()
')
AS t (
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
)
);
PERFORM dblink_disconnect('LINK1');
RETURN TRUE ;
END
$$ LANGUAGE plpgsql;
Mba hanangonana snapshots dia ampiasaina ny serivisy systemd sy script roa:
pg_current_activity.service
# /etc/systemd/system/pg_current_activity.service
[Unit]
Description=Collect history of pg_stat_activity , pg_locks
Wants=pg_current_activity.timer
[Service]
Type=forking
StartLimitIntervalSec=0
ExecStart=/home/postgres/pgutils/demon/get_current_activity.sh 10.124.70.40 postgres postgres
[Install]
WantedBy=multi-user.target
pg_current_activity.timer
# /etc/systemd/system/pg_current_activity.timer
[Unit]
Description=Run pg_current_activity.sh every 1 second
Requires=pg_current_activity.service
[Timer]
Unit=pg_current_activity.service
OnCalendar=*:*:0/1
AccuracySec=1
[Install]
WantedBy=timers.target
Noho izany, ny tantaran'ny fomba fijery dia angonina amin'ny endrika sary faharoa isaky ny segondra. Mazava ho azy, raha avela ho toy izao ny zava-drehetra, dia hitombo haingana be ny latabatra ary ho lasa tsy azo atao ny asa mamokatra bebe kokoa.
Ilaina ny mandamina ny fitahirizana angon-drakitra.
3. Fitahirizana ny tantara
Ho an'ny fitahirizana dia ampiasaina ny arsiva* latabatra voazarazara.
Fizarana vaovao no noforonina isan'ora, raha esorina amin'ny tabilao tantara* ny angona taloha, ka tsy miova firy ny haben'ny tabilao tantara* ary tsy miharatsy ny hafainganam-pandehan'ny fampidirana rehefa mandeha ny fotoana.
Ny famoronana fizarana vaovao dia ataon'ny plpgsql function activity_hist.archive_current_activity. Ny algorithm amin'ny asa dia tena tsotra (mampiasa ny ohatry ny fizarana ho an'ny latabatra archive_pg_stat_activity).
Mamorona sy mameno fizarana vaovao
EXECUTE format(
'CREATE TABLE ' || partition_name ||
' PARTITION OF activity_hist.archive_pg_stat_activity FOR VALUES FROM ( %L ) TO ( %L ) ' ,
to_char(date_trunc('year', partition_min_range ),'YYYY')||'-'||
to_char(date_trunc('month', partition_min_range ),'MM')||'-'||
to_char(date_trunc('day', partition_min_range ),'DD')||' '||
to_char(date_trunc('hour', partition_min_range ),'HH24')||':00',
to_char(date_trunc('year', partition_max_range ),'YYYY')||'-'||
to_char(date_trunc('month', partition_max_range ),'MM')||'-'||
to_char(date_trunc('day', partition_max_range ),'DD')||' '||
to_char(date_trunc('hour', partition_max_range ),'HH24')||':00'
);
INSERT INTO activity_hist.archive_pg_stat_activity
(
SELECT *
FROM activity_hist.history_pg_stat_activity
WHERE timepoint BETWEEN partition_min_range AND partition_max_range
);
Mamorona index
EXECUTE format (
'CREATE INDEX '||index_name||
' ON '||partition_name||' ( wait_event_type , backend_type , timepoint )'
);
EXECUTE format ('CREATE INDEX '||index_name||
' ON '||partition_name||' ( wait_event_type , backend_type , timepoint , queryid )'
);
Esory ny angona taloha amin'ny tabilao history_pg_stat_activity
DELETE
FROM activity_hist.history_pg_stat_activity
WHERE timepoint < partition_max_range;
Mazava ho azy, tsindraindray, ny fizarana taloha dia voafafa ho tsy ilaina.
Tatitra fototra
Raha ny marina, nahoana izany rehetra izany no atao? Mba hahazoana tatitra tsy dia mazava loatra mampahatsiahy ny Oracle's AWR.
Zava-dehibe ny manampy fa mba hahazoana tatitra dia mila manangana fifandraisana eo amin'ny pg_stat_activity sy pg_stat_statements ianao. Ny tabilao dia ampifandraisina amin'ny fampidirana tsanganana 'queryid' amin'ny tabilao 'history_pg_stat_activity', 'archive_pg_stat_activity'. Ny fomba fampidirana sandan'ny tsanganana dia mihoatra ny sahan'ity lahatsoratra ity ary voalaza eto β pg_stat_statements + pg_stat_activity + loq_query = pg_ash? .
TOTAL FOTOANA CPU HO AN'NY FANONTANIANA
fangatahana :
WITH hist AS
(
SELECT
aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND backend_type = 'client backend' AND datname != 'postgres' AND ( aa.wait_event_type IS NULL ) ANDaa.state = 'active'
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND backend_type = 'client backend' AND datname != 'postgres' AND ( ha.wait_event_type IS NULL )AND ha.state = 'active'
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
WITH hist AS
(
SELECT
aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( aa.wait_event_type IS NOT NULL )
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( ha.wait_event_type IS NOT NULL )
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
WITH hist AS
(
SELECT
aa.wait_event_type , aa.wait_event
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
aa.wait_event IS NOT NULL
GROUP BY aa.wait_event_type , aa.wait_event
UNION
SELECT
ha.wait_event_type , ha.wait_event
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
ha.wait_event IS NOT NULL
GROUP BY ha.wait_event_type , ha.wait_event
)
SELECT wait_event_type , wait_event
FROM hist
GROUP BY wait_event_type , wait_event
ORDER BY 1 ASC,2 ASC
----------------------------------------------------------------------
WITH hist AS
(
SELECT
aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( aa.wait_event_type = waitings_stat_rec.wait_event_type AND aa.wait_event = waitings_stat_rec.wait_event )
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( ha.wait_event_type = waitings_stat_rec.wait_event_type AND ha.wait_event = waitings_stat_rec.wait_event )
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
SELECT
MIN(date_trunc('second',timepoint)) AS started ,
count(*) * interval '1 second' as duration ,
pid , blocking_pids , relation , mode , locktype
FROM
activity_hist.archive_locking al
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY pid , blocking_pids , relation , mode , locktype
UNION
SELECT
MIN(date_trunc('second',timepoint)) AS started ,
count(*) * interval '1 second' as duration ,
pid , blocking_pids , relation , mode , locktype
FROM
activity_hist.history_locking
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY pid , blocking_pids , relation , mode , locktype
ORDER BY 1
SELECT
blocking_pids
FROM
activity_hist.archive_locking al
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY blocking_pids
UNION
SELECT
blocking_pids
FROM
activity_hist.history_locking
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY blocking_pids
ORDER BY 1
---------------------------------------------------------------
SELECT
pid , usename , application_name , datname ,
MIN(date_trunc('second',timepoint)) as started ,
count(*) * interval '1 second' as duration ,
state ,
query
FROM activity_hist.archive_pg_stat_activity
WHERE pid= current_pid AND
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour')
GROUP BY pid , usename , application_name ,
datname ,
state_change,
state ,
query
UNION
SELECT
pid , usename , application_name , datname ,
MIN(date_trunc('second',timepoint)) as started ,
count(*) * interval '1 second' as duration ,
state ,
query
FROM activity_hist.history_pg_stat_activity_for_reports
WHERE pid= current_pid AND
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour')
GROUP BY pid , usename , application_name ,
datname ,
state_change,
state ,
query
ORDER BY 5 , 1
Ny fanontaniana fototra aseho sy ny tatitra aterak'izany dia efa manamora ny fiainana rehefa mamakafaka ny zava-nitranga.
Miorina amin'ny fanontaniana fototra, afaka mahazo tatitra mitovy amin'ny AWR an'ny Oracle ianao. Ohatra amin'ny tatitra famintinana
+------------------------------------------------ ----------------------------------- | TATAO ATAO HO AN'NY ASA ATAO SY NY FIANDRASANA.
Mbola hitohy. Manaraka izany dia ny famoronana tantara hidin-trano (pg_stat_locks), famaritana amin'ny antsipiriany kokoa ny fizotran'ny famenoana latabatra.