Blogs

TRAG R&A-Finding Existing Queries Using Query and Varying Search Parameters

By Alan Hwang posted 05-25-2021 05:25 PM

  

Find existing queries that we can use is our goal.

Do you know how to find an existing similar query before creating a new one from scratch? Doing the research before creating a new one is a MUST approach that you should remember and always to do first. The SQL text of PS query is not stored as it is in system, but is generated when you click on the tab of View SQL in Query Manager. If you have access privilege in Query Manager, you can find all detail information about how the query is constructed. Although you may have privilege to view the public query, it is still very time consuming to go through each query to figure out which query that you can use, enhance or modify. Here are some tips and tricks that can help you find some queries that you want to see.

First thing you need to check: What is the output RECORD.FIELD of the query will return?

You must know the end result in terms of RECORD.FIELD. The end result in PS query is related to the RECORD.FIELD in TABLE.COLUMN in database tables. How can you find what the RECORD.FIELD are that the query will return? You can use the metadata PSQryField to find all RECORD.FIELDs that will be returned. The current existing query may return some of the RECORD.FIELDs which are interesting to you but may not include every RECORD.FIELDs that you want to see. The best scenario is that you find many of them which you want to get, so you can be pretty sure that the existing query can help you in some way during your research. This type of query is most likely what you can use directly or indirectly without creating a new one from scratch.

In this query below, we create a query allowing the user to do “List search” by specifying multiple RECORD.FIELD pairs separated by COMMA, then the query will find all potential queries that match any one of those RECORD.FIELD pairs. The user can start to investigate and may find something useful.

Without the nice query to find the useful existing queries, you still can try to find some by searching the metadata table PSQryField by yourself.

What are the records used in query to join?

Without knowing the exact RECORD.FIELD, you must know which RECORD are interesting to you at least. Do you know how to find an existing useful query by searching the records used in the query? The records that are used in query join represents some meaningful information associated with the output fields of the query. In this query example below, we let the user find all queries involved with those records. The more records used in query join means that the more specific type of information between records will be associated to each other. These types of information representing the PARENT-CHILD relationship between records. The PARENT-CHILD relationship will be joined together by specifying the PARENT-CHILD keys in query that is another kind of clues that we can use to search too. You can use the metadata PSQryRecord and PSQrySelect to find all records used and how they are joined together in query. From the example below, we can find more than 1000 LEDGER related queries were created by our university. Although the number is still too high to us, we have some idea to start with now at least.

 

What are the fields of record used in query to join?

The PARENT-CHILD keys are the RECORD.FIELDs representing the PARENT-CHILD relationship between database tables. Do you know how to find an existing similar query by searching the fields used in query? We need to specify at least one record in query and most of the time we need two records to join together in order to find the needed information from one PARENT record to the other CHILD relationship that represents the data we are searching for. In query join, the same meaningful field in both records are used to join together to find the PARENT-CHILD relation back from these two records, so we can limit the number of rows and only related rows should be returned from two records. This is the basic concept that the relational database ORACLE is designed. These PARENT-CHILD keys are extremely important in finding the existing queries that you need. You can use the metadata PSQryCriteria to find all the criteria used in query. In general, all the primary keys in parent record will be used to join with the corresponding keys of the child record. The name of these keys can be same or different, but the data underneath must be the same.

We have some customization here again. There is a good amount of work that needs to be done to make this raw data searchable and here is just some of the customized code showing the high level of design. For the users, they just use this searching query and they will find all the potential queries that they want to find. The idea of this searching query is allowing the user to specify up to 5 RECORD.FIELDs join keys used in query criteria of PS query.

 

 



What are the short and long description of the query?

After we have done all the searching above, we still can’t find any existing useful query. We can try something else below. Do you know how to find an existing similar query by searching the short or long description? The short and long description of query are stored in different fields that represent some meaningful information about the query. This is the easiest way to try to find some query but it is the most unreliable approach to find what you want, because the description is hard to predict in your searching. You can use the metadata PSQryDefn to find short and long description of the query.

 

 

Don’t forget about searching PS query expression too.

Here is another area that you can search but you may forget. Do you know how to find an existing similar query by searching the query expression in query? The query expression in PS query is a free form that allows developer to create some code used in PS query. The RECORD.FIELD name can be used in expression, so you need to search there too. Because of how the expression is saved in the database, we need to have some interpreter translating the PS Query expression data to a new form, so that the user can search the RECORD.FIELD from our searching query. The code below shows our customized solution on how we have that translation done in a view.

There is a new function SYSADM.UNC_ExpField_Decode( C.EXPRESSIONTEXT,  C.FIELDLIST,  C.FIELDLIST2) in code below, that will allow user to enter RECORD.FIELD searching string to query the tables, from metadata PSQRYEXPR A, PS_NC_EXP_FIELD C for matching the user input values. I can provide the code if you are interested to see how that code is written.

 

What’s the end result of your research?

What’s the end result of your research? It all depends on how good your search is utilized. The best case is finding one that can be used immediately. The second good choice is finding an existing one, then we can enhance the code by adding the PROMPTs, so that query can be used in multiple ways. The third scenario is finding one that can be modified and saved as a new query, so you don’t need to create a new one from scratch. The fourth case is researching more and more by specifying different searching criteria, so you may find something useful based on how much time and effort that you have spent, get some idea, and the worst case is creating a new one from scratch. The more you do the research, the better understanding of your system, and this process will help you in the long run. If you think that this is useful to your organization and we would love to share the code with you and good luck in your research.

 

4 comments
83 views

Permalink

Comments

11-05-2021 11:47 AM

Thank you very much, Alan! This is super helpful. I have the task of translating this from Oracle to SQL Server but I'm eager to get it working. :)

11-04-2021 07:33 AM

Hi Dana,
Here are the codes that you are looking for close to the end of comments. I have included all the codes mentioned in this Blog here. Please let me know if you are interested in this and would like to know more about it.
For the function UNC_ExpField_Decode , it is a recursive function and you can uncomment the comment codes to see how that works as design.

              

Here is the SQL statement for the PS Queries demostrate in this Blog:

SQL for first query: NC_DOC_PSQryField_List_Srch

SELECT A.OPRID, A.QRYNAME, A.RECNAME, COUNT(*)

  FROM PS_NC_QRYFLD_VW A

  WHERE ( ( ' ' = :1

     OR A.QRYNAME LIKE :1)

     AND ( ' ' = :2

     OR A.RECNAME LIKE :2)

     AND ( ' ' = :3

     OR A.FIELDNAME LIKE :3)

     AND ( instr(upper(replace(replace( replace(:4 || ',' , ' ', ''), ',,' , ','), ',,', ',') ),  A.RECNAME || '.' ||  A.FIELDNAME || ',' , 1) > '0'))

  GROUP BY  A.OPRID,  A.QRYNAME,  A.RECNAME

  HAVING ( count(*) = REGEXP_COUNT( replace(replace( replace(:4 || ',' , ' ', ''), ',,' , ','), ',,', ',') ,  ',', 1));

 

2nd query: NC_DOC_PSQRYRECORD   

Query SQL

              

SELECT A.OPRID, A.QRYNAME, A.SELNUM, A.RCDNUM, A.RECNAME, A.JOINTYPE, A.CORRNAME, B.DESCR, TO_CHAR(CAST((B.LASTUPDDTTM) AS TIMESTAMP),'YYYY-MM-DD-HH24.MI.SS.FF'), B.LASTUPDOPRID, B.DESCRLONG, TO_CHAR(CAST((B.CREATEDTTM) AS TIMESTAMP),'YYYY-MM-DD-HH24.MI.SS.FF'), B.EXECLOGGING, B.QRYFOLDER, B.QRYJOINOPTIMIZE, B.QRYTYPE, B.SELCOUNT, B.EXPCOUNT, B.BNDCOUNT

  FROM PSQRYRECORD A, PSQRYDEFN B

  WHERE ( ( ' ' = :1

     OR A.QRYNAME LIKE :1)

     AND ( ' ' = :2

     OR A.RECNAME LIKE :2)

     AND A.OPRID = B.OPRID

     AND A.QRYNAME = B.QRYNAME)

  ORDER BY 1, 2, 4, 3;

 

3rd Query SQL : NC_DOC_PSQryCriteria_List_Srch

              

SELECT B.OPRID2, B.QRYNAME, B.SELNUM, B.CRTNUM, B.COMBTYPE, B.DESCR100, B.DESCR254_MIXED, SYSADM.UNC_ExpField_Decode( B.EXPRESSIONTEXT,  B.FIELDLIST,  B.FIELDLIST2)

  FROM PS_NC_QRY_CRTEX_VW B

  WHERE ( ( ' ' = :1

     OR B.QRYNAME LIKE :1)

     AND ( ' ' = :2

     OR instr(upper(SYSADM.UNC_ExpField_Decode( B.DESCR254_MIXED,  B.FIELDLIST,  B.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,1,'i' ), ',', '') , 1) > '0'

     OR instr(upper(SYSADM.UNC_ExpField_Decode( B.DESCR254_MIXED,  B.FIELDLIST,  B.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,2,'i' ), ',', '') , 1) > 0

     OR instr(upper(SYSADM.UNC_ExpField_Decode( B.DESCR254_MIXED,  B.FIELDLIST,  B.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,3,'i' ), ',', '') , 1) > 0

     OR instr(upper(SYSADM.UNC_ExpField_Decode( B.DESCR254_MIXED,  B.FIELDLIST,  B.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,4,'i' ), ',', '') , 1) > '0'

     OR instr(upper(SYSADM.UNC_ExpField_Decode( B.DESCR254_MIXED,  B.FIELDLIST,  B.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,5,'i' ), ',', '') , 1) > '0')

     AND ( ' ' = :3

     OR B.OPRID2 LIKE :3));

 

4th      Query SQL : NC_Doc_PSQryExpr_List_Srch

              

SELECT A.OPRID, A.QRYNAME, A.EXPNUM, (case when  instr(upper(SYSADM.UNC_ExpField_Decode(   A.EXPRESSIONTEXT,    C.FIELDLIST,    C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || ':2' || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,1,'i' ), ',', '') , 1) > 0  then 'Y,' else 'N,' end)

||(case when  instr(upper(SYSADM.UNC_ExpField_Decode(   A.EXPRESSIONTEXT,    C.FIELDLIST,    C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || ':2' || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,2,'i' ), ',', '') , 1) > 0   then 'Y,' else 'N,' end)

||(case when  instr(upper(SYSADM.UNC_ExpField_Decode(   A.EXPRESSIONTEXT,    C.FIELDLIST,    C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || ':2' || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,3,'i' ), ',', '') , 1) > 0   then 'Y,' else 'N,' end)

||(case when  instr(upper(SYSADM.UNC_ExpField_Decode(   A.EXPRESSIONTEXT,    C.FIELDLIST,    C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || ':2' || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,4,'i' ), ',', '') , 1) > 0   then 'Y,' else 'N,' end)

||(case when  instr(upper(SYSADM.UNC_ExpField_Decode(   A.EXPRESSIONTEXT,    C.FIELDLIST,    C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || ':2' || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,5,'i' ), ',', '') , 1) > 0   then 'Y,' else 'N,' end), A.EXPRESSIONTEXT, SYSADM.UNC_ExpField_Decode( A.EXPRESSIONTEXT,  C.FIELDLIST,  C.FIELDLIST2), A.FIELDTYPE, A.LENGTH, A.DECIMALPOS, A.USECOUNT, C.FLDNUM, C.QRYFLDNAME, C.SELNUM, C.RECNAME, C.FIELDNAME, C.HEADING

  FROM PSQRYEXPR A, PS_NC_EXP_FIELD C

  WHERE ( ( ' ' = :1

     OR A.QRYNAME LIKE :1)

     AND ( ' ' = :2

     OR ( length( dbms_lob.substr(  A.EXPRESSIONTEXT, 3000,1) ) < 3000

     AND ( instr(upper(SYSADM.UNC_ExpField_Decode( A.EXPRESSIONTEXT,  C.FIELDLIST,  C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,1,'i' ), ',', '') , 1) > 0

     OR instr(upper(SYSADM.UNC_ExpField_Decode( A.EXPRESSIONTEXT,  C.FIELDLIST,  C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,2,'i' ), ',', '') , 1) > 0

     OR instr(upper(SYSADM.UNC_ExpField_Decode( A.EXPRESSIONTEXT,  C.FIELDLIST,  C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,3,'i' ), ',', '') , 1) > '0'

     OR instr(upper(SYSADM.UNC_ExpField_Decode( A.EXPRESSIONTEXT,  C.FIELDLIST,  C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,4,'i' ), ',', '') , 1) > '0'

     OR instr(upper(SYSADM.UNC_ExpField_Decode( A.EXPRESSIONTEXT,  C.FIELDLIST,  C.FIELDLIST2) ), replace( regexp_substr( replace(replace( replace(',' || :2 || ',' , ' ', ''), ',,' , ','), ',,', ',') , '[^,]+,' , 1,5,'i' ), ',', '') , 1) > '0')))

     AND ( ' ' = :3

     OR A.OPRID LIKE :3)

     AND A.OPRID = C.OPRID

     AND A.QRYNAME = C.QRYNAME

     AND C.FLDEXPNUM = A.EXPNUM)

  ORDER BY 4 DESC, 1, 2, 3;

 

UNC customized view NC_Exp_Field:

 

SELECT f.*,

       f.qryfldname AS field_value_char,

       replace(to_char(substr(e.expressiontext, 1, 3800)), '''', '''''') AS expressionText,

       fp.char_val AS fieldList,

       fp.field_val AS fieldList2

  FROM psqryfield f,

       psqryexpr e,

       (SELECT oprid,

               qryname,

               LISTAGG(fe.tok, ',') WITHIN GROUP(ORDER BY oprid, qryname, selnum DESC, fldnum DESC) || ',' AS char_val,

               LISTAGG(replace(fe.qryfldname, ',', ';'), ',') WITHIN GROUP(ORDER BY oprid, qryname, selnum DESC, fldnum DESC) || ',' AS field_val

          FROM (SELECT x.oprid,

                       x.qryname,

                       x.selnum,

                       x.fldnum,

                       x.qryfldname,

                       '%' || x.selnum || '.' || x.fldnum AS tok

                  FROM psqryfield x,

                       (SELECT oprid, qryname

                          FROM psqryfield

                         GROUP BY oprid, qryname

                        HAVING SUM(LENGTH(qryfldname || ',')) < 3800) y

                 WHERE x.oprid = y.oprid

                   AND x.qryname = y.qryname

                   AND instr(x.qryfldname, '''') <= 0

                 ORDER BY x.oprid, x.qryname, x.selnum DESC, x.fldnum DESC) fe

         GROUP BY oprid, qryname) fp

 WHERE /* debugging: fp.qryname = 'NC_OSR_PROJ_INVOICE_EXPS_BUD1' AND */

 f.fldrcdnum = 0

 AND f.oprid = fp.oprid

 AND f.qryname = fp.qryname

 AND f.oprid = e.oprid

 AND f.qryname = e.qryname

 AND f.fldexpnum = e.expnum

 AND instr(substr(e.expressiontext, 1, 3800), '%') > 1 /* Limit the total length of qryfiled to be

less than 3800 to prevent overflow in string */

;

 

 

UNC customized function UNC_ExpField_Decode :

create or replace function sysadm.UNC_ExpField_Decode ( inExp IN varchar2, inCode IN varchar2, inReplace IN varchar2)

return varchar2 is

       v_found number := 0;

       v_pos_repl number := 0;

       v_pos_code number := 0;

       v_inExp varchar2(4000) := inExp;

       v_inCode varchar2(4000) := inCode;

       v_inReplace varchar2(4000) := inReplace;

 

  begin

    v_found := instr(v_inExp, '%') ;

      v_pos_code := instr(v_inCode, ',', 1);

      v_pos_repl := instr(v_inReplace, ',', 1);

 

    if instr(v_inExp, '%') > 0 and instr(v_inCode, ',', 1) > 0 and instr(v_inReplace, ',', 1) > 0 then

      -- debugging : dbms_output.put_line('test v_pos_code' ||  v_pos_code || ' ' || substr( v_inCode,  1, instr(v_inCode, ',',1)-1) || ' v_repl ' || v_pos_repl || ' ' || substr(v_inReplace, 1,  instr(v_inReplace, ',', 1)-1) );

      return  UNC_ExpField_Decode( replace( v_inExp, substr( v_inCode,  1, instr(v_inCode, ',',1)-1), substr(v_inReplace,  1,instr(v_inReplace, ',', 1)-1) ),

                             substr( v_inCode,  instr(v_inCode, ',',1)+1 ), substr(v_inReplace,  instr(v_inReplace, ',', 1)+1) )

                                       ;

    end if;

    return v_inExp;

  end;

 

UNC customized view NC_Qry_Crt_VW:  

SELECT b.oprid AS oprid2,

       b.qryname,

       b.selnum,

       b.crtnum,

       b.combtype,

       decode(b.exprtype,

              0,

              'Effective Sequence',

              1,

              'Const',

              2,

              'Field',

              3,

              'Expr',

              4,

              'Subquery',

              5,

              'List',

              6,

              'Current Date',

              7,

              'Tree',

              8,

              'Prompt',

              9,

              'Const-Const',

              10,

              'Const-Field',

              11,

              'Const-Expr',

              12,

              'Field-Const',

              13,

              'Field-Field',

              14,

              'Field-Expr',

              15,

              'Expr-Const',

              16,

              'Expr-Field',

              17,

              'Expr-Expr') AS DESCR100,

       to_char(decode(b.lparenlvl, 1, '(', '') ||

               decode(FL.fldrcdnum,

                      0,

                      substr(dL.expressiontext, 1, 20),

                      FL.qryFldName) ||

               decode(b.condtype,

                      2,

                      ' = ',

                      3,

                      ' <> ',

                      4,

                      ' > ',

                      5,

                      ' not greater than ',

                      6,

                      ' < ',

                      7,

                      ' not less than ',

                      8,

                      ' in ',

                      9,

                      ' Not in List ',

                      10,

                      ' between ',

                      11,

                      ' Not Between ',

                      12,

                      ' exists ',

                      13,

                      ' Does Not Exist ',

                      14,

                      ' Like ',

                      15,

                      ' Not Like ',

                      16,

                      ' Is Null ',

                      17,

                      ' Is Not Null ',

                      18,

                      ' In Tree ',

                      19,

                      ' Not in Tree ',

                      20,

                      ' Eff Date <=  ') ||

               (CASE

                  WHEN b.exprtype IN (1) THEN

                   ''''

                  ELSE

                   ''

                END) || nvl(FR.QryFldName,

                            CASE

                              WHEN b.exprtype IN (1, 2, 3, 8, 9, 10, 11, 15, 16, 17) THEN

                               substr(EL.expressiontext, 1, 250)

                              WHEN b.condtype IN (8, 9, 18, 19) THEN

                               substr(EL.expressiontext, 1, 250)

                              ELSE

                               substr(dL.expressiontext, 1, 100)

                            END) || (CASE

                                       WHEN b.condtype IN (10, 11) THEN

                                        ' and ' || (CASE

                                          WHEN b.exprType IN (9, 11, 12, 14, 15, 17) THEN

                                           to_char(substr(ER.expressiontext, 1, 50))

                                          ELSE

                                           FR2.QryFldName

                                        END)

                                     END) ||

               (CASE

                  WHEN b.exprtype IN (1) THEN

                   ''''

                  ELSE

                   ''

                END) || decode(b.rparenlvl, 1, ')', '')) AS DESCR254_MIXED,

       FL.fldRcdNum AS FLDNUMVAL1,

       FR.fldrcdnum AS FLDNUMVAL2,

       51 AS FLDNUMVAL3,

       b.negation,

       b.lparenlvl,

       b.lcrtselnum,

       b.lcrtfldnum,

       b.condtype,

       b.exprtype,

       b.r1crtselnum,

       b.r1crtfldnum,

       b.r1crtexpnum,

       b.r2crtselnum,

       b.r2crtfldnum,

       b.r2crtexpnum,

       b.rparenlvl,

       b.qryojselnum,

       substr(FLE.expressiontext, 1, 50) AS DESCR100A,

       substr(dL.expressiontext, 1, 50) AS DESCR100B,

       substr(EL.expressiontext, 1, 50) AS DESCR100C,

       substr(ER.expressiontext, 1, 50) AS DESCR100D,

       FR.QryFldName AS DESCR100E,

       FR2.QryFldName AS DESCR100F

  FROM psqryfield    FL,

       psqrycriteria b,

       psqryfield    FR,

       psqryexpr     EL,

       psqryexpr     ER,

       psqryfield    FR2,

       psqryexpr     FLE,

       psqryexpr     dL

 WHERE FL.qryname = b.qryname

   AND FL.Oprid = b.Oprid

   AND FL.fldnum = b.lcrtfldnum

   AND b.lcrtselnum = FL.selnum

   AND FL.qryname = dL.qryname(+)

   AND FL.Oprid = dL.Oprid(+)

   AND FL.fldnum = dL.expnum(+)

   AND FL.qryname = FLE.qryname(+)

   AND FL.Oprid = FLE.Oprid(+)

   AND FL.fldExpnum = FLE.expnum(+)

   AND b.qryname = FR.qryname(+)

   AND b.Oprid = FR.Oprid(+)

   AND b.r1crtfldnum = FR.fldnum(+)

   AND b.r1crtselnum = FR.selnum(+)

   AND b.qryname = FR2.qryname(+)

   AND b.Oprid = FR2.Oprid(+)

   AND b.r2crtfldnum = FR2.fldnum(+)

   AND b.r2crtselnum = FR2.selnum(+)

   AND b.qryname = EL.qryname(+)

   AND b.Oprid = EL.Oprid(+)

   AND b.r1crtexpnum = EL.expnum(+)

   AND b.qryname = ER.qryname(+)

   AND b.Oprid = ER.Oprid(+)

   AND b.r2crtexpnum = ER.expnum(+)

 ORDER BY b.oprid, b.qryname, b.selnum, b.crtnum

;

11-03-2021 06:15 PM

I'm unable to find the contents of your UNC_ExpField_Decode function in your presentations. Are you able to provide this? Thanks in advance.

10-28-2021 04:26 PM

Hi Alan. I heard in a presentation of yours that you might be willing to share the queries and code you created. I'd be very interested in receiving a copy of this!