6

Pyramid Usage Model and Analysis

Using Pyramid to analyze itself.

The following document describes how to build a data model using Pyramid’s database repository as a data source to be able to analyze content usage and performance in Pyramid itself. 


The resulting model will give you insights into: 

  1. WHO: Query transactions executed by users and tenants
  2. WHEN: Query transactions by time - down to the hour and minute level
  3. WHAT: Query transactions by data source server and type, database and model, with further details related to content items, the main content folders, specific hierarchies and measures used in each query (from the “elements” hierarchies)
  4. WHERE: Query transactions by processing server
  5. WHICH: Query transactions by which types of content 

And of course, any combination of the above. 

Setup

  1. Flow:
    1. Start a new advanced data model in Pyramid. Add a relevant data source (PostgreSQL, MS SQL Server, Oracle), depending on which type of repository you have elected to deploy
      1. From the database listing in the property cards, select the Pyramid database repository
      2. From the table selection, choose two tables
        • Transactionview view
        • Server_log_transaction_columns table
      3. Click Add Selected Nodes, to add these tables and views to the flow
    2. Add a new Convert Columns node after the transactionview select node in the flow from the Column Operations side menu
      1. Change the date column to type “date”, with format yyyy-mm-dd
    3. Add a new In-memory target database node and provide a name for the database (“Pyramid Usage”)
  2. Model:
    1. Change the default measure to “Total Time” on the configuration tab
    2. On the Tables tab:
      1. Rename the server_log_transaction_columns to “Elements”
      2. Rename the transactionview table to “Events”
      3. Make sure the Elements table is inner joined to the Events table using the “transaction_id” column
      4. Change all the measures in the Events table to “Average” aggregations: column_count, row,count, cell_count, connection_time, post_query_time, pre_query_time, other_time, query_time, total_time
      5. Right-click on transaction_id column and add a count measure
      6. Optional: Right-click on user_name and add a distinct count measure (to count unique users). 
  3. Save and then Process your model.  

The resulting model will take a snapshot of your transactional activity in Pyramid for further analysis.
Set up a schedule on this model to reprocess the database once a day, typically off-peak.  

Metrics

Response Size Metrics

  • Column Count: The number of RAW columns in the result set
  • Row Count: The number of RAW rows in the result set
  • Cell count = columns x rows

The RAW result set is not always indicative of the drawn result in the visual.
For instance, matrix grids can be much larger drawn than their underlying RAW results.

Time Metrics

  •    Pre-query: Pyramid engine time before the query is submitted.
  •    Post-query: Pyramid engine time after the query is submitted.
  •    Other: any time spent on scripting logic and dynamic functions as part of the query
  • Server: pre-query + post-query + other
  •    Connection: the time taken to connect to the data source
  •    Query: the time data source takes to respond to the query
  • Query Engine: Connection + Queries and Calculations
  • Total: server + query engine + network + client

The time aggregates may be slightly off because:

  • the measurements are in millisecond integers - so rounding is ignored.
  • not every time element is always fully additive - because some items may run in parallel to each other (like 'connection' and 'other' times)

Using these metrics, you should be able to spot performance issues in your query designs vs data source performance vs Pyramid's processing performance.

  

3replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Hi Itamar,

     

    This is very useful feature. But I have one question regarding this. The view is returning only data for last 4 months. Is there any way we can extend this period?

     

    Thanks,

    Uroš

    Reply Like
  • Hi Uroš,
     

    The Usage Model is based on the transactional data that is stored in the Pyramid database repository.
    This data is stored in the db for a limited time which Administrators can change if needed.

    To configure Pyramid to keep the data for a longer period of time, you can change the "Keep Transactions Logs For These Days" setting in the Admin Console > Settings > Logging.

    Before making this change, please make sure the database is capable of storing that much information.


     

    Reply Like
  • For explanations on the model metrics see this posting.

    https://community.pyramidanalytics.com/t/184nyf/pyramid-usage-model

    Reply Like
Like6 Follow
  • 6 Likes
  • 11 days agoLast active
  • 3Replies
  • 160Views
  • 4 Following