Skip to content
Advertisement

Postgres Function not working when I have a large result

I’m copying information from table 1(tmp_subtype) to table 2(subtype_user). I have a test table 1 with 15 registers. I run this function into postgres:

CREATE OR REPLACE FUNCTION VERIFY_AND_INSERT_SUPTYPE()
    RETURNS text AS $$
    DECLARE
       register_subtype RECORD;
       existe INT DEFAULT 0;
       MESSAGE_EXCEPTION TEXT;
       cursor_subtype CURSOR
       FOR
          SELECT tsd.subtype,tsd.type_id_client,tsd.id_client,tsd.email
          FROM tmp_subtype tsd;
    BEGIN

      OPEN cursor_subtype;
      LOOP
         FETCH cursor_subtype INTO register_subtype;
         EXIT WHEN NOT FOUND;

         SELECT COUNT(*) INTO existe FROM (
            SELECT sdu.id_client FROM subtype_user sdu
            WHERE sdu.id_client = register_subtype.id_client AND sdu.type_id_client = register_subtype.type_id_client
            LIMIT 1
         ) SUB0;

         IF existe = 0 THEN
            INSERT INTO subtype_user(subtype,type_id_client,id_client,email)
            VALUES (register_subtype.subtype,register_subtype.type_id_client,register_subtype.id_client,register_subtype.email);
         ELSE
            UPDATE subtype_user sdu2 SET subtype=register_subtype.subtype,email=register_subtype.email
            WHERE sdu2.id_client = register_subtype.id_client AND sdu2.type_id_client = register_subtype.type_id_client;
         END IF;
      END LOOP;
      CLOSE cursor_subtype;

      RETURN 'OK';

      EXCEPTION WHEN OTHERS THEN
            GET STACKED DIAGNOSTICS MESSAGE_EXCEPTION = MESSAGE_TEXT;
            RETURN MESSAGE_EXCEPTION;
    END; $$
    LANGUAGE plpgsql;

It works, but When I run this function with the real table 1, it is not working. The function finishes but nothing happend. The real table 1 has 1 million of registers.

Advertisement

Answer

Row-by-row processing with embedded counting is a recipe for slow and inefficient processing. Additionally your check for existence won’t work if the function is invoked from concurrent transactions. As far as I can tell you can replace the whole loop and cursor with a single INSERT statement:

CREATE OR REPLACE FUNCTION VERIFY_AND_INSERT_SUPTYPE()
    RETURNS text 
AS $$
DECLARE
  MESSAGE_EXCEPTION TEXT;
BEGIN

  INSERT INTO subtype_user(subtype, type_id_client, id_client, email)
  SELECT tsd.subtype, tsd.type_id_client, tsd.id_client, tsd.email
  FROM tmp_subtype tsd
  ON conflict (id_client, type_id_client) DO UPDATE 
    SET  subtype = excluded.register_subtype,
         email = excluded.email;

  RETURN 'OK';

  EXCEPTION WHEN OTHERS THEN
        GET STACKED DIAGNOSTICS MESSAGE_EXCEPTION = MESSAGE_TEXT;
        RETURN MESSAGE_EXCEPTION;
END; $$
LANGUAGE plpgsql;

I probable would not add an exception handler to begin with, so that the caller sees the complete exception.

2 People found this is helpful
Advertisement