PS Query & SQL

 View Only
  • 1.  Auditing Queries that output sensitive data

    Posted 11-05-2025 10:49 AM

    Hello HEUG Friends!

    I am working on a project to audit all queries that contain sensitive data, such as DOB, SS#, Banking Information, etc. I have a few questions and hope all you awesome people can help.

    • How would you go about finding all the sensitive fields?
    • How do you determine if the record is public or private? I saw a query type of 1 or 0, but for some reason everything comes back as 1 for me, even though I'm using the correct record and field.

    I have identified the records to query, but this creates a lot of false positives because a record may be used and its fields may exist, even if the field isn't being presented. 

    • How would you find the outputs, as a field and as an expression?
    • How would you find data sources used in things like BI-Publisher for example?

    Any help and a push in the right direction will help me.

    THANK YOU!



    ------------------------------
    Lidia Anderson
    Manager, Campus Solutions
    Central Washington University
    lidia.anderson@cwu.edu
    ------------------------------
    Alliance 2026 Registration is Open!


  • 2.  RE: Auditing Queries that output sensitive data

    Posted 11-05-2025 11:01 AM

    Query access to tables is on PSQRYACCLSTRECS  (Query Access Record List) by Permission List, as long as you have enabled "update query list cache." And update regularly. Then join to PSROLECLASS to PSROLEUSER to PSOPRDEFN to get the users. Otherwise, you have to query the query security trees and not at easy. lol

     

    Private queries are identified by OPRID field being populated on table PSQRYDEFN. Null means public query.

     

    To pull column headings, and tables/fields for a query. This has a prompt for query name.

     

    SELECT A.QRYNAME, A.COLUMNNUM, A.RECNAME, A.FIELDNAME, DECODE( A.HDGTYPE, 3,  C.SHORTNAME, '4',  C.LONGNAME,  A.HEADING), A.HDGTYPE
      FROM ((PSQRYFIELD A LEFT OUTER JOIN  PSPTRECFLDLBLVW B ON  A.RECNAME = B.RECNAME AND A.FIELDNAME = B.FIELDNAME ) LEFT OUTER JOIN  PSDBFLDLABL C ON  C.FIELDNAME = B.FIELDNAME AND C.LABEL_ID = B.LABEL_ID )
      WHERE ( A.QRYNAME LIKE :1
         AND A.OPRID = ' '
         AND A.SELNUM = 1
         AND A.COLUMNNUM <> 0)
      ORDER BY 2

    I use the following to find fields used in criteria or expressions. In this instance, was looking for Admit Type values.

     

     

    SELECT A.QRYNAME, A.DESCR, A.OPRID, TO_CHAR(CAST((D.LASTEXECDTTM) AS TIMESTAMP),'YYYY-MM-DD-HH24.MI.SS.FF'), TO_CHAR(CAST((A.LASTUPDDTTM) AS TIMESTAMP),'YYYY-MM-DD-HH24.MI.SS.FF'), A.LASTUPDOPRID, B.OPRDEFNDESC, E.R1CRTEXPNUM, F.EXPRESSIONTEXT, G.FIELDNAME, E.CONDTYPE
      FROM PSQRYDEFN A, PSOPRDEFN B, PSQRYRECORD C, PSQRYSTATS D, PSQRYCRITERIA E, PSQRYEXPR F, PSQRYFIELD G
      WHERE ( A.LASTUPDOPRID =  B.OPRID(+)
         AND A.OPRID = C.OPRID
         AND A.QRYNAME = C.QRYNAME
         AND A.OPRID = D.OPRID
         AND A.QRYNAME = D.QRYNAME
         AND C.OPRID = E.OPRID
         AND C.QRYNAME = E.QRYNAME
         AND C.SELNUM = E.SELNUM
         AND E.OPRID = F.OPRID
         AND E.QRYNAME = F.QRYNAME
         AND E.R1CRTEXPNUM = F.EXPNUM
         AND E.OPRID = G.OPRID
         AND E.QRYNAME = G.QRYNAME
         AND E.SELNUM = G.SELNUM
         AND E.LCRTFLDNUM = G.FLDNUM
         AND ( SUBSTR( F.EXPRESSIONTEXT, 1, 500) LIKE '%FYR%'
         OR SUBSTR( F.EXPRESSIONTEXT, 1, 500) LIKE '%TRN%'
         OR SUBSTR( F.EXPRESSIONTEXT, 1, 500) LIKE '%FYT%'))

     

     

    Here is query that shows if a user can see a query, based on if the user can every table in the query. It has union to pull the tables the user cannot see. Prompts for user and query name.

     

    SELECT DISTINCT A.QRYNAME, 'Access to table', A.RECNAME, E.OPRID, E.OPRDEFNDESC, A.OPRID
      FROM PSQRYRECORD A, PSQRYACCLSTRECS B, PSROLECLASS C, PSROLEUSER D, PSOPRDEFN E
      WHERE ( A.QRYNAME LIKE :1
         AND A.RECNAME = B.RECNAME
         AND C.CLASSID = B.CLASSID
         AND C.ROLENAME = D.ROLENAME
         AND D.ROLEUSER = E.OPRID
         AND E.OPRID = :2
         AND A.QRYNAME NOT IN (SELECT K.QRYNAME
      FROM PSQRYRECORD K
      WHERE K.QRYNAME LIKE :1
         AND K.RECNAME NOT IN (SELECT L.RECNAME
      FROM PSRECDEFN L)))
    UNION
    SELECT F.QRYNAME, 'NO access to table', F.RECNAME, ':2' , ':1' , F.OPRID
      FROM PSQRYRECORD F
      WHERE ( F.QRYNAME LIKE :1
         AND F.RECNAME NOT IN (SELECT G.RECNAME
      FROM PSQRYACCLSTRECS G, PSROLECLASS H, PSROLEUSER I, PSOPRDEFN J
      WHERE H.CLASSID = G.CLASSID
         AND H.ROLENAME = I.ROLENAME
         AND I.ROLEUSER = J.OPRID
         AND J.OPRID = :2)
         AND F.QRYNAME NOT IN (SELECT M.QRYNAME
      FROM PSQRYRECORD M
      WHERE M.QRYNAME LIKE :1
         AND M.RECNAME NOT IN (SELECT N.RECNAME
      FROM PSRECDEFN N)))
      ORDER BY 1

     

     

     

     

     

    Stephen Forrest

     

    Institutional Research Analyst the Third

    University Decision Support

    Enrollment Management Research Group

    Office of the Provost

    Southern Methodist University

    sforrest@smu.edu

    214-768-4015

     

    "The only reason we put the data in, is to take the data out."

     

     

     

     




    Alliance 2026 Registration is Open!


  • 3.  RE: Auditing Queries that output sensitive data

    Posted 11-06-2025 01:11 PM

    Hi Lidia, 

    My fantastic colleague @Scott Douglas did an Alliance presentation on this topic a few years back.  You can find the recording here.  I know that he has done something like what you are attempting (with different fields) so hopefully he will add some context here for you.

    As for why you're only getting 1, best guess would be a PS Query "secret" security join, where it's automatically restricting query results based on your oprid, but I don't know for sure without know what records you're querying.



    ------------------------------
    Scott Nishizaki
    Connected Campus Community of Practice
    Developer/Analyst
    Azusa Pacific University
    ------------------------------

    Message from the HEUG Marketplace:
    ------------------------------
    Find, Review, and Engage with Higher Education-focused solution providers, products, and services using the HEUG Marketplace.
    ------------------------------

    Alliance 2026 Registration is Open!


  • 4.  RE: Auditing Queries that output sensitive data

    Posted 11-07-2025 08:28 AM

    Hi,

    Public queries are queries in Query Manager where the OPRID is a single space.

    -Alexei

    https://blog.psftdba.com/2024/02/what-psquery-is-that.html



    ------------------------------
    Alexei Tetenov
    Software Engineer
    Rochester Institute of Technology
    ------------------------------

    Message from the HEUG Marketplace:
    ------------------------------
    Find, Review, and Engage with Higher Education-focused solution providers, products, and services using the HEUG Marketplace.
    ------------------------------

    Alliance 2026 Registration is Open!


  • 5.  RE: Auditing Queries that output sensitive data

    Posted 11-07-2025 08:41 AM
    Hi - 

    QRYTYPE is referring to the "Query Type" field which you can select on via the advanced search record display

    ...and view within the Properties link in the query.  

    The value of 1 refers to "User".

    David Ehrlich | Senior Business System Analyst

    Duke University | Student Information Services & System (SISS)

    david.ehrlich@duke.edu | 919-684-1206

    www.sissoffice.duke.edu





    Alliance 2026 Registration is Open!


  • 6.  RE: Auditing Queries that output sensitive data

    Posted 11-07-2025 12:52 PM
      |   view attached

    Thanks, @Scott Nishizaki, for the shout-out.

    There are already some excellent answers here about how to query the metadata, and that can get you most of the way. For instance, finding the fields should be easy using Stephen Forrest's first SQL.

    Expressions will be harder, as the field references are replaced by some numeric references that aren't easy to translate back. I ran across an intriguing presentation about how UNC built a "Metadata Detective Tool" that looks like it might help make investigations like this easier, but I haven't taken the time to watch it yet. 

    The presentation "other Scott" is referring to spends a lot of time talking about all kinds of metadata tables, but then goes into the basics of a tool we developed at Azusa Pacific to export full SQL text for each PS Query into a table to allow for more easily searching the text for instances of hard-coded values, etc. With this tool, you could do the following:

    • Run an export of all query SQL to a table (ours is called PS_APU_QRY_SQL_EXP)
    • Write a query as follows:
      • Use PSQRYDEFN to rule out disabled queries, get the Query Type, etc., then
      • Use PSQRYRECORD to filter to queries that use the sensitive record(s) and get the CORRNAME used as the table alias (e.g., the ""A" in "A.EMPLID").
      • Use PSRECFIELDALL to find all fields associated with that record, and optionally you could filter to just the fields you care about
      • Use your Query export table to look for mentions of [Alias].[Fieldname]
    SELECT A.OPRID
      , A.QRYNAME
      , DECODE(A.OPRID,' ','Public','') PUBLIC_QRY
      , B.CORRNAME
      , B.RECNAME
      , C.FIELDNAME
    FROM PSQRYDEFN A
    JOIN PSQRYRECORD B
      ON A.OPRID = B.OPRID
      AND A.QRYNAME = B.QRYNAME
    JOIN PSRECFIELDALL C
      ON B.RECNAME = C.RECNAME
    JOIN PS_APU_QRY_SQL_EXP D
      ON A.OPRID = D.APU_OPRID
      AND A.QRYNAME = D.QRYNAME
    WHERE A.QRYDISABLED <> 'Y'
      AND B.RECNAME IN ('SENSITIVE_REC1', 'SENSITIVE_REC2')
      AND C.FIELDNAME IN ('SSN', 'GPA', 'OTHER_SENSITIVE_FIELD')
      AND D.QRYSQL LIKE '%' || B.CORRNAME || '.' || C.FIELDNAME || '%'

    This still isn't perfect, as it will find any reference to these fields, including those just in joins and criteria... but it could be a good start! If I were doing this research and didn't have time to build an even more perfect tool for the job, I would first run Stephen Forrest's SQL to find the offending fields, then run this and perhaps filter out the queries that I already looked at from the first list. That would give a smaller list of queries to investigate for sensitive expressions.

    Also, if you're recreating my SQL above in a PS Query, you'll probably need to use a fancy trick on the last line of criteria where you say expression 'x' is equal to:   'x' AND D.QRYSQL LIKE '%' || B.CORRNAME || '.' || C.FIELDNAME || '%'

    Good luck!



    ------------------------------
    Scott Douglas
    Sr. Business Systems Analyst, Student Information Systems
    Azusa Pacific University
    ------------------------------

    Alliance 2026 Registration is Open!