ProHoster > Blog > Administration > Studie af implementering af forretningslogik pÄ niveau med gemte funktioner i PostgreSQL

Studie af implementering af forretningslogik pÄ niveau med gemte funktioner i PostgreSQL

Inspiration for writing the study came from the article ‘During quarantine, the load increased fivefold, but we were ready.’ How Lingualeo migrated to PostgreSQL with 23 million users. I also found the article published four years ago interesting — Implementation of business logic in MySQL.

What struck me was that the same idea-"implementing business logic in the database".

Studie af implementering af forretningslogik pÄ niveau med gemte funktioner i PostgreSQL

occurred to me as well.

I also wanted to preserve, for myself primarily, the interesting developments that arose during implementation. Especially considering that a strategic decision was recently made to change the architecture and move business logic to the backend level. So, everything that was developed will soon be of no use to anyone and nobody will find it interesting.

The methods described are not any kind of revelation or exceptional know how, everything is traditional and has been implemented many times (for example, I applied a similar approach 20 years ago on Oracle). I just decided to gather everything in one place. Maybe someone will find it useful. As practice shows, quite often the same idea comes to different people independently. And it’s useful to leave it for myself as a memory.

Of course, nothing in this world is perfect; mistakes and typos are unfortunately possible. Criticism and comments are always welcome and expected. And one more small detail — specific implementation details are omitted. After all, everything is still being used in a real working project. So, the article serves as a study and description of the general concept, nothing more. I hope the details are sufficient for understanding the overall picture.

The overall idea is — ‘divide and conquer, hide and own’

The idea is classic — a separate schema for tables, a separate schema for stored functions.
The client does not have direct access to the data. All the client can do is call a stored function and process the received response.

Roller

CREATE ROLE store;


CREATE ROLE sys_functions;


CREATE ROLE loc_audit_functions;


CREATE ROLE service_functions;


CREATE ROLE business_functions;

Schemas

Table storage schema

Target tables that implement subject entities.

CREATE SCHEMA store AUTHORIZATION store;

Schema of system functions

System functions, particularly for logging changes in tables.

CREATE SCHEMA sys_functions AUTHORIZATION sys_functions;

Local audit schema

Funktioner og tabeller til implementering af lokal revision af udfÞrelsen af gemte funktioner og Êndring af mÄl-tabeller.

Opret skema loc_audit_functions BEVILLING loc_audit_functions;

Schema for servicefunktioner

Funktioner til service- og DML-funktioner.

Opret skema service_functions BEVILLING service_functions;

Schema for forretningsfunktioner

Funktioner til slutbrugerens forretningsfunktioner.

Opret skema business_functions BEVILLING business_functions;

Adgangsrettigheder

Rolle — DBA har fuld adgang til alle skemaer (adskilt fra rollen DB Owner).

Opret rolle dba_role;
Giv store til dba_role;
Giv sys_functions til dba_role;
Giv loc_audit_functions til dba_role;
Giv service_functions til dba_role;
Giv business_functions til dba_role;

Rolle — USER har privilegium EXECUTE i skemaet business_functions.

Opret rolle user_role;

Privilegier mellem skemaer

GRANT
Da alle funktioner oprettes med attributten SECURITY DEFINER er det nþdvendigt med en instruktion FJERN EXECUTE PÅ ALLE FUNKTIONER
 FRA public;

FJERN EXECUTE PÅ ALLE FUNKTIONER I SKEMA sys_functions FRA public ; 
FJERN EXECUTE PÅ ALLE FUNKTIONER I SKEMA loc_audit_functions FRA public ; 
FJERN EXECUTE PÅ ALLE FUNKTIONER I SKEMA service_functions FRA public ; 
FJERN EXECUTE PÅ ALLE FUNKTIONER I SKEMA business_functions FRA public ; 

Giv BRUG PÅ SKEMA sys_functions TIL dba_role ; 
Giv EXECUTE PÅ ALLE FUNKTIONER I SKEMA sys_functions TIL dba_role ;
Giv BRUG PÅ SKEMA loc_audit_functions TIL dba_role ; 
Giv EXECUTE PÅ ALLE FUNKTIONER I SKEMA loc_audit_functions TIL dba_role ;
Giv BRUG PÅ SKEMA service_functions TIL dba_role ; 
Giv EXECUTE PÅ ALLE FUNKTIONER I SKEMA service_functions TIL dba_role ;
Giv BRUG PÅ SKEMA business_functions TIL dba_role ; 
Giv EXECUTE PÅ ALLE FUNKTIONER I SKEMA business_functions TIL dba_role ;
Giv EXECUTE PÅ ALLE FUNKTIONER I SKEMA business_functions TIL user_role ;

Giv ALLE PRIVILEGIER PÅ SKEMA store TIL GRUPPE business_functions ;
Giv ALLE PRIVILEGIER PÅ ALLE TABELLER I SKEMA store TIL business_functions ;
Giv BRUG PÅ ALLE SEKVENSER I SKEMA store TIL business_functions ;

SĂ„ skemaet for databasen er klar. Vi kan begynde at fylde det med data.

MĂ„l-tabeller

Oprettelsen af tabeller er triviel. Ingen sÊrlige forholdsregler, udover at det er besluttet at undgÄ brug af SERIAL og generere sekvenser eksplicit. Desuden, selvfÞlgelig maksimalt brug af instruktionen

KOMMENTAR PÅ ...

Kommentarer til af alle objekter, uden undtagelser.

Lokal revision

Til registrering af udfÞrelsen af gemte funktioner og Êndring af mÄl-tabeller anvendes en lokal revisions-tabel, der inkluderer detaljer om klientforbindelsen, etiketten pÄ det kaldte modul, samt de faktiske vÊrdier af ind- og udgangsparametre i form af JSON.

Systemfunktioner

Er beregnet til at logge Êndringer i mÄl-tabeller. De er trigger-funktioner.

Skabelon — systemfunktion

---------------------------------------------------------
-- INSERT
CREATE OR REPLACE FUNCTION sys_functions.table_insert_log ()
RETURNS TRIGGER AS $$
BEGIN
  PERFORM loc_audit_functions.make_log( ' '||'table' , 'insert' , json_build_object('id', NEW.id)  );
  RETURN NULL ;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;

CREATE TRIGGER table_after_insert AFTER INSERT ON storage.table FOR EACH ROW EXECUTE PROCEDURE sys_functions.table_insert_log();

---------------------------------------------------------
-- UPDATE
CREATE OR REPLACE FUNCTION sys_functions.table_update_log ()
RETURNS TRIGGER AS $$
BEGIN
  IF OLD.column != NEW.column
  THEN
    PERFORM loc_audit_functions.make_log( ' '||'table' , 'update' , json_build_object('OLD.column', OLD.column , 'NEW.column' , NEW.column )  );
  END IF ;
  RETURN NULL ;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;

CREATE TRIGGER table_after_update AFTER UPDATE ON storage.table FOR EACH ROW EXECUTE PROCEDURE sys_functions.table_update_log ();

---------------------------------------------------------
-- DELETE
CREATE OR REPLACE FUNCTION sys_functions.table_delete_log ()
RETURNS TRIGGER AS $$
BEGIN
  PERFORM loc_audit_functions.make_log( ' '||'table' , 'delete' , json_build_object('id', OLD.id )  );
  RETURN NULL ;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;

CREATE TRIGGER table_after_delete AFTER DELETE ON storage.table FOR EACH ROW EXECUTE PROCEDURE sys_functions.table_delete_log ();

Servicefunktioner

Designet til at implementere service- og DML-operationer pÄ target-tabeller.

Skabelon — servicefunktion

--INSERT
--RETURN id AF NY RAD
CREATE OR REPLACE FUNCTION service_functions.table_insert ( new_column store.table.column%TYPE )
RETURNS integer AS $$
DECLARE
  new_id integer ;
BEGIN
  -- Generer ny id
  new_id = nextval('store.table.seq');

  -- IndsĂŠt i tabellen
  INSERT INTO store.table
  ( 
    id ,
    column
   )
  VALUES
  (
   new_id ,
   new_column
   );

RETURN new_id ;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;

--DELETE
--RETURN ANTAL RADER SLETTET
CREATE OR REPLACE FUNCTION service_functions.table_delete ( current_id integer ) 
RETURNS integer AS $$
DECLARE
  rows_count integer  ;    
BEGIN
  DELETE FROM store.table WHERE id = current_id; 

  GET DIAGNOSTICS rows_count = ROW_COUNT;                                                                           

  RETURN rows_count ;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;
 
-- OPDATER DETALJER
-- RETURN ANTAL RADER OPDATERET
CREATE OR REPLACE FUNCTION service_functions.table_update_column 
(
  current_id integer 
  ,new_column store.table.column%TYPE
) 
RETURNS integer AS $$
DECLARE
  rows_count integer  ; 
BEGIN
  UPDATE  store.table
  SET
    column = new_column
  WHERE id = current_id;

  GET DIAGNOSTICS rows_count = ROW_COUNT;                                                                           

  RETURN rows_count ;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;

Forretningsfunktioner

Designet til slutbrugsforretningsfunktioner kaldet af klienten. Returnerer altid — JSON. For at intercept og logge kþrselsfejl, anvendes blokken EXCEPTION.

Skabelon — forretningsfunktion

CREATE OR REPLACE FUNCTION business_functions.business_function_template(
--Input parameters        
 )
RETURNS JSON AS $$
DECLARE
  ------------------------
  --for exception catching
  error_message text ;
  error_json json ;
  result json ;
  ------------------------ 
BEGIN
--LOGGING
  PERFORM loc_audit_functions.make_log
  (
    'business_function_template',
    'STARTED',
    json_build_object
    (
	--IN Parameters
    ) 
   );

  PERFORM business_functions.notice('business_function_template');            

  --START BUSINESS PART
  --END BUSINESS PART

  -- SUCCESFULLY RESULT
  PERFORM business_functions.notice('result');
  PERFORM business_functions.notice(result);

  PERFORM loc_audit_functions.make_log
  (
    'business_function_template',
    'FINISHED', 
    json_build_object( 'result',result )
  );

  RETURN result ;
----------------------------------------------------------------------------------------------------------
-- EXCEPTION CATCHING
EXCEPTION                        
  WHEN OTHERS THEN    
    PERFORM loc_audit_functions.make_log
    (
      'business_function_template',
      'STARTED',
      json_build_object
      (
	--IN Parameters	
      ) , TRUE );

     PERFORM loc_audit_functions.make_log
     (
       'business_function_template',
       ' ERROR',
       json_build_object('SQLSTATE',SQLSTATE ), TRUE 
     );

     PERFORM loc_audit_functions.make_log
     (
       'business_function_template',
       ' ERROR',
       json_build_object('SQLERRM',SQLERRM  ), TRUE 
      );

     GET STACKED DIAGNOSTICS error_message = RETURNED_SQLSTATE ;
     PERFORM loc_audit_functions.make_log
     (
      'business_function_template',
      ' ERROR-RETURNED_SQLSTATE',json_build_object('RETURNED_SQLSTATE',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = COLUMN_NAME ;
     PERFORM loc_audit_functions.make_log
     (
       'business_function_template',
       ' ERROR-COLUMN_NAME',
       json_build_object('COLUMN_NAME',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = CONSTRAINT_NAME ;
     PERFORM loc_audit_functions.make_log
     (
      'business_function_template',
      ' ERROR-CONSTRAINT_NAME',
      json_build_object('CONSTRAINT_NAME',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = PG_DATATYPE_NAME ;
     PERFORM loc_audit_functions.make_log
     (
       'business_function_template',
       ' ERROR-PG_DATATYPE_NAME',
       json_build_object('PG_DATATYPE_NAME',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = MESSAGE_TEXT ;
     PERFORM loc_audit_functions.make_log
     (
       'business_function_template',
       ' ERROR-MESSAGE_TEXT',json_build_object('MESSAGE_TEXT',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = SCHEMA_NAME ;
     PERFORM loc_audit_functions.make_log
     (s
       'business_function_template',
       ' ERROR-SCHEMA_NAME',json_build_object('SCHEMA_NAME',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = PG_EXCEPTION_DETAIL ;
     PERFORM loc_audit_functions.make_log
     (
      'business_function_template',
      ' ERROR-PG_EXCEPTION_DETAIL',
      json_build_object('PG_EXCEPTION_DETAIL',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = PG_EXCEPTION_HINT ;
     PERFORM loc_audit_functions.make_log
     (
       'business_function_template',
       ' ERROR-PG_EXCEPTION_HINT',json_build_object('PG_EXCEPTION_HINT',error_message  ), TRUE );

     GET STACKED DIAGNOSTICS error_message = PG_EXCEPTION_CONTEXT ;
     PERFORM loc_audit_functions.make_log
     (
      'business_function_template',
      ' ERROR-PG_EXCEPTION_CONTEXT',json_build_object('PG_EXCEPTION_CONTEXT',error_message  ), TRUE );                                      

    RAISE WARNING 'ALARM: %' , SQLERRM ;

    SELECT json_build_object
    (
      'isError' , TRUE ,
      'errorMsg' , SQLERRM
     ) INTO error_json ;

  RETURN  error_json ;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;

Resultat

For an overview, I think this is quite sufficient. If anyone is interested in the details and results, please leave a comment, and I will gladly add more details.

P.S.

Logging a simple error - type of input parameter

-[ RECORD 1 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1072
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          | STARTED
jsonb_pretty    | {
                |     "dko": {
                |         "id": 4,
                |         "type": "Type1",
                |         "title": "OPRETTET AF addKD",
                |         "Weight": 10,
                |         "Tr": "300",
                |         "reduction": 10,
                |         "isTrud": "TRUE",
                |         "description": "beskrivelse",
                |         "lowerTr": "100",
                |         "measurement": "mÄling1",
                |         "methodology": "m1",
                |         "passportUrl": "filer",
                |         "upperTr": "200",
                |         "weightingFactor": 100.123,
                |         "actualTrValue": null,
                |         "upperTrCalcNumber": "120"
                |     },
                |     "CardId": 3
                | }
-[ RECORD 2 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1073
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL
jsonb_pretty    | {
                |     "SQLSTATE": "22P02"
                | }
-[ RECORD 3 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1074
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL
jsonb_pretty    | {
                |     "SQLERRM": "ugyldigt input syntaks for type numeric: "null""
                | }
-[ RECORD 4 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1075
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-RETURNED_SQLSTATE
jsonb_pretty    | {
                |     "RETURNED_SQLSTATE": "22P02"
                | }
-[ RECORD 5 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1076
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-KOLON_NAVN
jsonb_pretty    | {
                |     "COLUMN_NAME": ""
                | }
-[ RECORD 6 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1077
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-BETINGELSE_NAVN
jsonb_pretty    | {
                |     "CONSTRAINT_NAME": ""
                | }
-[ RECORD 7 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1078
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-PG_DATATYPE_NAVN
jsonb_pretty    | {
                |     "PG_DATATYPE_NAME": ""
                | }
-[ RECORD 8 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1079
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-MEDDELELSE_TEXT
jsonb_pretty    | {
                |     "MESSAGE_TEXT": "ugyldigt input syntaks for type numeric: "null""
                | }
-[ RECORD 9 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1080
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-SKEMA_NAVN
jsonb_pretty    | {
                |     "SCHEMA_NAME": ""
                | }
-[ RECORD 10 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1081
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-PG_UNDTAGNING_DETALJE
jsonb_pretty    | {
                |     "PG_EXCEPTION_DETAIL": ""
                | }
-[ RECORD 11 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1082
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-PG_UNDTAGNING_HINT
jsonb_pretty    | {
                |     "PG_EXCEPTION_HINT": ""
                | }
-[ RECORD 12 ]-
date_trunc      | 2020-08-19 13:15:46
id              | 1083
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-PG_UNDTAGNING_KONTEXT
jsonb_pretty    | {
usename         | emp1
log_module      | addKD
log_module_hash | 0b4c1529a89af3ddf6af3821dc790e8a
status          |  FEJL-MEDDELELSE_TEXT
jsonb_pretty    | {
                |     "MESSAGE_TEXT": "ugyldigt input syntaks for type numeric: "null""
                | }

Kilde: habr.com

KĂžb pĂ„lidelig webhosting med DDoS-beskyttelse, VPS VDS-servere đŸ”„ KĂžb pĂ„lidelig webhosting med DDoS-beskyttelse, VPS VDS-servere | ProHoster