Teko ea ho theha analogue ea ASH bakeng sa PostgreSQL
Mokhoa oa bothata
Ho ntlafatsa lipotso tsa PostgreSQL, bokhoni ba ho sekaseka nalane ea ts'ebetso, haholoholo, ho leta, linotlolo, le lipalo tsa tafole li hlokahala haholo.
pgsentinel extension :
«Lintlha tsohle tse bokelletsoeng li bolokoa ho RAM feela, 'me palo ea memori e sebelisitsoeng e laoloa ke palo ea lirekoto tsa ho qetela tse bolokiloeng.
Ho kenyelelitsoe karolo ea queryid - potso e ts'oanang ho tsoa ho katoloso ea pg_stat_statements (ho hlokahala hore e kenyelloe pele).«
Sena, ehlile, se ka thusa haholo, empa ntho e thata ka ho fetisisa ke ntlha ea pele. ”Lintlha tsohle tse bokelletsoeng li bolokiloe feela ho RAM ”, ke. ho na le tšusumetso holim'a motheo oa sepheo. Ho phaella moo, ha ho na histori ea senotlolo le lipalo-palo tsa tafole. Tseo. tharollo ka kakaretso ha e ea fella: “Ha ho na sephutheloana se seng se lokiselitsoe ho kenya. Ho khothaletsoa hore u khoasolle mehloli le ho bokella laebrari ka bouena. Pele o hloka ho kenya sephutheloana sa "devel" bakeng sa seva sa hau mme o behe tsela ea ho pg_config ho PATH e fapaneng.".
Ka kakaretso, ho na le likhang tse ngata, 'me tabeng ea li-database tse tebileng tsa tlhahiso, ho ka' na ha se ke ha khoneha ho etsa letho ka seva. Re hloka ho tla ka ntho ea rona hape.
Temoso
Ka lebaka la molumo o moholo haholo le ka lebaka la nako ea tlhahlobo e sa fellang, sengoloa ke sa mofuta oa tlhaiso-leseling, ho fapana le sete sa lintlha le liphetho tsa mahareng.
Boitsebiso bo eketsehileng bo tla lokisoa hamorao, ka likarolo
Litlhoko tsa moralo bakeng sa tharollo
Hoa hlokahala ho theha sesebelisoa se u lumellang ho boloka:
pg_stat_activity histori ea pono Nalane ea senotlolo sa nako u sebelisa pono ea pg_locks
Tlhokahalo ea tharollo- fokotsa tšusumetso ho database eo u batlang ho e etsa.
Kakanyo e akaretsang- moemeli oa pokello ea lintlha ha a hlahisoe sebakeng sa polokelo ea litaba, empa sebakeng sa polokelo ea litaba e le tšebeletso ea systemd. E, tahlehelo e 'ngoe ea data e ka khoneha, empa sena ha se bohlokoa bakeng sa ho tlaleha, empa ha ho na tšusumetso ho database ea sepheo ho latela sebaka sa memori le disk. 'Me tabeng ea ho sebelisa letamo la khokahanyo, phello ea mekhoa ea mosebedisi e fokola haholo.
Mekhahlelo ea ho phethahatsa
1.Litafole tsa litšebeletso
Ho sebelisoa schema e fapaneng ho boloka litafole, e le hore ho se ke ha rarahana ho hlahlojoa ha litafole tse kholo tse sebelisoang.
DROP SCHEMA IF EXISTS activity_hist ;
CREATE SCHEMA activity_hist AUTHORIZATION monitor ;
Bohlokoa: Leano ha lea etsoa ka har'a polokelo ea litaba tse reriloeng, empa le bobolokelong ba ho beha leihlo.
pg_stat_activity histori ea pono
Tafole e sebelisoa ho boloka linepe tsa hajoale tsa pono ea pg_stat_activity
activity_hist.history_pg_stat_activity :
--ACTIVITY_HIST.HISTORY_PG_STAT_ACTIVITY
DROP TABLE IF EXISTS activity_hist.history_pg_stat_activity;
CREATE TABLE activity_hist.history_pg_stat_activity
(
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text ,
queryid bigint
);
Ho potlakisa ho kenya - ha ho li-index kapa lithibelo.
Ho boloka nalane ka boeona, ho sebelisoa tafole e arohaneng:
activity_hist.archive_pg_stat_activity :
DROP TABLE IF EXISTS activity_hist.archive_pg_stat_activity;
CREATE TABLE activity_hist.archive_pg_stat_activity
(
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text ,
queryid bigint
)
PARTITION BY RANGE (timepoint);
Kaha tabeng ena ha ho na litlhoko tsa lebelo la ho kenya, li-index tse ling li bōpiloe ho potlakisa ho thehoa ha litlaleho.
Nalane e thibelang nako
Tafole e sebelisoa ho boloka linepe tsa hajoale tsa linotlolo tsa nako:
activity_hist.history_locking :
--ACTIVITY_HIST.HISTORY_LOCKING
DROP TABLE IF EXISTS activity_hist.history_locking;
CREATE TABLE activity_hist.history_locking
(
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
);
Hape, ho potlakisa ho kenya, ha ho na li-index kapa lithibelo.
Ho boloka nalane ka boeona, ho sebelisoa tafole e arohaneng:
activity_hist.archive_locking:
DROP TABLE IF EXISTS activity_hist.archive_locking;
CREATE TABLE activity_hist.archive_locking
(
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
)
PARTITION BY RANGE (timepoint);
Kaha tabeng ena ha ho na litlhoko tsa lebelo la ho kenya, li-index tse ling li bōpiloe ho potlakisa ho thehoa ha litlaleho.
2.Ho tlatsa nalane ea hajoale
Ho bokella linepe tsa pono ka kotloloho, ho sebelisoa bash script e tsamaisang mosebetsi oa plpgsql.
plgsql Ts'ebetso ea dblink e fihlella lipono sebakeng sa polokelo ea litaba ebe e kenya mela ka har'a litafole tsa lits'ebeletso sebakeng sa polokelo ea litaba.
get_current_activity.sql
CREATE OR REPLACE FUNCTION activity_hist.get_current_activity( current_host text , current_s_name text , current_s_pass text ) RETURNS BOOLEAN AS $$
DECLARE
database_rec record;
dblink_str text ;
BEGIN
EXECUTE 'SELECT dblink_connect(''LINK1'',''host='||current_host||' port=5432 dbname=postgres'||
' user='||current_s_name||' password='||current_s_pass|| ' '')';
--------------------------------------------------------------------
--GET pg_stat_activity stats
INSERT INTO activity_hist.history_pg_stat_activity
(
SELECT * FROM dblink('LINK1',
'SELECT
now() ,
datid ,
datname ,
pid ,
usesysid ,
usename ,
application_name ,
client_addr ,
client_hostname ,
client_port ,
backend_start ,
xact_start ,
query_start ,
state_change ,
wait_event_type ,
wait_event ,
state ,
backend_xid ,
backend_xmin ,
query ,
backend_type
FROM pg_stat_activity
')
AS t (
timepoint timestamp without time zone ,
datid oid ,
datname name ,
pid integer,
usesysid oid ,
usename name ,
application_name text ,
client_addr inet ,
client_hostname text ,
client_port integer,
backend_start timestamp with time zone ,
xact_start timestamp with time zone ,
query_start timestamp with time zone ,
state_change timestamp with time zone ,
wait_event_type text ,
wait_event text ,
state text ,
backend_xid xid ,
backend_xmin xid ,
query text ,
backend_type text
)
);
---------------------------------------
--ACTIVITY_HIST.HISTORY_LOCKING
INSERT INTO activity_hist.history_locking
(
SELECT * FROM dblink('LINK1',
'SELECT
now() ,
lock.locktype,
lock.relation,
lock.mode,
lock.transactionid as tid,
lock.virtualtransaction as vtid,
lock.pid,
pg_blocking_pids(lock.pid),
lock.granted
FROM pg_catalog.pg_locks lock LEFT JOIN pg_catalog.pg_database db ON db.oid = lock.database
WHERE NOT lock.pid = pg_backend_pid()
')
AS t (
timepoint timestamp without time zone ,
locktype text ,
relation oid ,
mode text ,
tid xid ,
vtid text ,
pid integer ,
blocking_pids integer[] ,
granted boolean
)
);
PERFORM dblink_disconnect('LINK1');
RETURN TRUE ;
END
$$ LANGUAGE plpgsql;
Ho bokella linepe tsa pono, ho sebelisoa ts'ebeletso ea systemd le lingoloa tse peli:
pg_current_activity.service
# /etc/systemd/system/pg_current_activity.service
[Unit]
Description=Collect history of pg_stat_activity , pg_locks
Wants=pg_current_activity.timer
[Service]
Type=forking
StartLimitIntervalSec=0
ExecStart=/home/postgres/pgutils/demon/get_current_activity.sh 10.124.70.40 postgres postgres
[Install]
WantedBy=multi-user.target
pg_current_activity.timer
# /etc/systemd/system/pg_current_activity.timer
[Unit]
Description=Run pg_current_activity.sh every 1 second
Requires=pg_current_activity.service
[Timer]
Unit=pg_current_activity.service
OnCalendar=*:*:0/1
AccuracySec=1
[Install]
WantedBy=timers.target
Fana ka litokelo ho lingoliloeng:
# chmod 755 pg_current_activity.timer
# chmod 755 pg_current_activity.service
Ha re qale tšebeletso:
# systemctl daemon-reload
# systemctl qala pg_current_activity.service
Kahoo, histori ea maikutlo e bokelloa ka mokhoa oa li-snapshots tsa bobeli ka bobeli. Ha e le hantle, haeba ntho e 'ngoe le e' ngoe e sala e le joalo, litafole li tla eketseha ka potlako ka boholo 'me mosebetsi o fokolang kapa o fokolang o ke ke oa khoneha.
Hoa hlokahala ho hlophisa polokelo ea data.
3. Ho boloka histori
Bakeng sa ho boloka, ho sebelisoa li-archive tsa litafole tse arohaneng.
Li-partitions tse ncha li bōptjoa hora e 'ngoe le e' ngoe, ha data ea khale e tlosoa litafoleng tsa histori *, kahoo boholo ba litafole tsa histori * ha bo fetohe haholo 'me lebelo la ho kenya ha le fokotsehe ka nako.
Theho ea likarolo tse ncha e etsoa ke plpgsql function activity_hist.archive_current_activity. Algorithm ea mosebetsi e bonolo haholo (ho sebelisa mohlala oa karolo bakeng sa tafole ea archive_pg_stat_activity).
Theha le ho tlatsa karolo e ncha
EXECUTE format(
'CREATE TABLE ' || partition_name ||
' PARTITION OF activity_hist.archive_pg_stat_activity FOR VALUES FROM ( %L ) TO ( %L ) ' ,
to_char(date_trunc('year', partition_min_range ),'YYYY')||'-'||
to_char(date_trunc('month', partition_min_range ),'MM')||'-'||
to_char(date_trunc('day', partition_min_range ),'DD')||' '||
to_char(date_trunc('hour', partition_min_range ),'HH24')||':00',
to_char(date_trunc('year', partition_max_range ),'YYYY')||'-'||
to_char(date_trunc('month', partition_max_range ),'MM')||'-'||
to_char(date_trunc('day', partition_max_range ),'DD')||' '||
to_char(date_trunc('hour', partition_max_range ),'HH24')||':00'
);
INSERT INTO activity_hist.archive_pg_stat_activity
(
SELECT *
FROM activity_hist.history_pg_stat_activity
WHERE timepoint BETWEEN partition_min_range AND partition_max_range
);
Ho theha li-index
EXECUTE format (
'CREATE INDEX '||index_name||
' ON '||partition_name||' ( wait_event_type , backend_type , timepoint )'
);
EXECUTE format ('CREATE INDEX '||index_name||
' ON '||partition_name||' ( wait_event_type , backend_type , timepoint , queryid )'
);
E tlosa lintlha tsa khale ho tloha ho history_pg_stat_activity table
DELETE
FROM activity_hist.history_pg_stat_activity
WHERE timepoint < partition_max_range;
Ha e le hantle, nako le nako, likarolo tsa khale li tlosoa e le tse sa hlokahaleng.
Litlaleho tsa motheo
Ha e le hantle, ke hobane’ng ha see sohle se etsoa? Ho fumana litlaleho tse hopotsang Oracle's AWR ka mokhoa o hlakileng.
Ho bohlokoa ho eketsa hore ho fumana litlaleho, o hloka ho theha khokahano lipakeng tsa pg_stat_activity le pg_stat_statements maikutlo. Litafole li hokahantsoe ka ho kenyelletsa kholomo ea 'queryid' ho litafole tsa 'history_pg_stat_activity', 'archive_pg_stat_activity'. Mokhoa oa ho eketsa boleng ba kholomo o feta boholo ba sengoloa sena mme o hlalositsoe mona - pg_stat_statements + pg_stat_activity + loq_query = pg_ash? .
KAOFELA EA NAKO EA CPU EA LIPOTSO
Kopo :
WITH hist AS
(
SELECT
aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND backend_type = 'client backend' AND datname != 'postgres' AND ( aa.wait_event_type IS NULL ) ANDaa.state = 'active'
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND backend_type = 'client backend' AND datname != 'postgres' AND ( ha.wait_event_type IS NULL )AND ha.state = 'active'
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
WITH hist AS
(
SELECT
aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( aa.wait_event_type IS NOT NULL )
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( ha.wait_event_type IS NOT NULL )
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
WITH hist AS
(
SELECT
aa.wait_event_type , aa.wait_event
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
aa.wait_event IS NOT NULL
GROUP BY aa.wait_event_type , aa.wait_event
UNION
SELECT
ha.wait_event_type , ha.wait_event
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
ha.wait_event IS NOT NULL
GROUP BY ha.wait_event_type , ha.wait_event
)
SELECT wait_event_type , wait_event
FROM hist
GROUP BY wait_event_type , wait_event
ORDER BY 1 ASC,2 ASC
----------------------------------------------------------------------
WITH hist AS
(
SELECT
aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid ,
count(*) * interval '1 second' AS duration
FROM activity_hist.archive_pg_stat_activity aa
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( aa.wait_event_type = waitings_stat_rec.wait_event_type AND aa.wait_event = waitings_stat_rec.wait_event )
GROUP BY aa.wait_event_type , aa.wait_event , aa.query ,aa.queryid
UNION
SELECT
ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid,
count(*) * interval '1 second' AS duration
FROM activity_hist.history_pg_stat_activity_for_reports ha
WHERE timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
backend_type = 'client backend' AND datname != 'postgres' AND
( ha.wait_event_type = waitings_stat_rec.wait_event_type AND ha.wait_event = waitings_stat_rec.wait_event )
GROUP BY ha.wait_event_type , ha.wait_event , ha.query ,ha.queryid
)
SELECT query , queryid , SUM( duration ) as duration
FROM hist
GROUP BY query , queryid
ORDER BY 3 DESC
SELECT
MIN(date_trunc('second',timepoint)) AS started ,
count(*) * interval '1 second' as duration ,
pid , blocking_pids , relation , mode , locktype
FROM
activity_hist.archive_locking al
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY pid , blocking_pids , relation , mode , locktype
UNION
SELECT
MIN(date_trunc('second',timepoint)) AS started ,
count(*) * interval '1 second' as duration ,
pid , blocking_pids , relation , mode , locktype
FROM
activity_hist.history_locking
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY pid , blocking_pids , relation , mode , locktype
ORDER BY 1
SELECT
blocking_pids
FROM
activity_hist.archive_locking al
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY blocking_pids
UNION
SELECT
blocking_pids
FROM
activity_hist.history_locking
WHERE
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour') AND
NOT granted AND
locktype = 'relation'
GROUP BY blocking_pids
ORDER BY 1
---------------------------------------------------------------
SELECT
pid , usename , application_name , datname ,
MIN(date_trunc('second',timepoint)) as started ,
count(*) * interval '1 second' as duration ,
state ,
query
FROM activity_hist.archive_pg_stat_activity
WHERE pid= current_pid AND
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour')
GROUP BY pid , usename , application_name ,
datname ,
state_change,
state ,
query
UNION
SELECT
pid , usename , application_name , datname ,
MIN(date_trunc('second',timepoint)) as started ,
count(*) * interval '1 second' as duration ,
state ,
query
FROM activity_hist.history_pg_stat_activity_for_reports
WHERE pid= current_pid AND
timepoint BETWEEN pg_stat_history_begin+(current_hour_diff * interval '1 hour') AND pg_stat_history_end+(current_hour_diff * interval '1 hour')
GROUP BY pid , usename , application_name ,
datname ,
state_change,
state ,
query
ORDER BY 5 , 1
Mohlala:
------------------------------------------------- ------------------------------------------------- ------------------------------------------------- ---------------------------------------------------------------------------------------------------------------- --------------------------------------------------- ------+------------------------------------------ - -------+---------------------------------------- | #| pid| usename| kopo_lebitso| lebitso | qadile| nako| naha| potso +----------+---------+----------+-----+ --------- -------------------------------------------------- -------------------------------------------- ------- ----------------- | 1| 26211| seseng| psql| tdb1| 2019-09-02 19:31:54| 00:00:04 | e se nang letho| | 2 | 26211| seseng| psql| tdb1| 2019-09-02 19:31:58| 00:00:06 | e sa sebetseng khoebong| QALA; | 3 | 26211| seseng| psql| tdb1| 2019-09-02 19:32:16| 00:01:45 | e sa sebetseng khoebong| lock table wafer_data; | 4 | 26211| seseng| psql| tdb1| 2019-09-02 19:35:54| 00:01:23 | e se nang letho| itlama; | 5 | 26211| seseng| psql| tdb1| 2019-09-02 19:38:46| 00:00:02 | e sa sebetseng khoebong| QALA; | 6 | 26211| seseng| psql| tdb1| 2019-09-02 19:38:54| 00:00:08 | e sa sebetseng khoebong| lock table wafer_data; | 7 | 26211| seseng| psql| tdb1| 2019-09-02 19:39:08| 00:42:42 | e se nang letho| itlama; | 8 | 26211| seseng| psql| tdb1| 2019-09-03 07:12:07| 00:00:52 | e sebetsang| khetha test_del();
Tsoelo-pele.
Lipotso tsa mantlha tse bontšitsoeng le litlaleho tse hlahisitsoeng li se li ntse li nolofatsa bophelo ha ho hlahlojoa liketsahalo tsa ts'ebetso.
Ho ipapisitsoe le lipotso tsa mantlha, u ka fumana tlaleho e ts'oanang hantle le Oracle's AWR. Kakaretso ea mohlala oa tlaleho
+------------------------------------------------- ------------------------------------ | TLALEHO E KOPANENG EA KETSAHALO LE HO LETA.
E tla ntšetsoa pele. E latelang moleng ke ho theha nalane ea senotlolo (pg_stat_locks), tlhaloso e qaqileng haholoanyane ea mokhoa oa ho tlatsa litafole.