Increasing Timeout Settings in Nginx Reverse Proxy

If you're using Nginx as a reverse proxy, follow these steps to increase the timeout settings:

1. Open your nginx.conf file, typically located in /etc/nginx/nginx.conf or inside your site's configuration file under /etc/nginx/sites-available/.

2. Locate or add the following directives inside the server or location block:

   proxy_connect_timeout 600s;
   proxy_send_timeout 600s;
   proxy_read_timeout 600s;
   send_timeout 600s;

3. Save the file and restart Nginx to apply the changes:

   sudo systemctl restart nginx

This increases the timeout settings to 10 minutes (600 seconds), reducing the chances of timeout-related 502 and 504 errors.

Increasing Timeout Settings in IIS

If you are using IIS, follow these steps to increase the timeout:

1. Open IIS Manager on the Pyramid web servers.

2. Double-click on Application Request Router Cache.

3. Click on Server Proxy Settings.

4. Locate the timeout settings and double the current value (or increase it to at least 10 minutes).

5. Click Apply to save the changes.

Increasing Timeout Settings for an AWS Load balancer

🔗 For the most up-to-date information on AWS ALB settings, visit:
👉 AWS ALB Documentation
👉 AWS Health Checks

1. Increase ALB Idle Timeout

• Default: 60 seconds
• Recommended: Increase to 600 seconds

Steps:

1. Go to EC2 Dashboard → Load Balancers
2. Select your Application Load Balancer (ALB)
3. Go to the Attributes tab
4. Click Edit
5. Set Idle timeout to 600 seconds
6. Click Save

2. Adjust Target Group Health Check Settings

• Increase timeout & interval to prevent premature failures

Steps:

1. Go to EC2 Dashboard → Target Groups
2. Select your Target Group
3. Go to the Health Checks tab → Click Edit
4. Set:
   • Timeout: 10 seconds
   • Interval: 30 seconds
   • Unhealthy threshold: 5
   • Healthy threshold: 3
5. Click Save

Other Web Servers and Load Balancers

If you are using any other third-party web server or load balancer, please refer to its documentation on how to increase its timeouts.

By increasing these timeout settings, you can ensure that your Pyramid application remains stable and minimizes 504/502 errors caused by timeouts.

Pyramid stops running tasks such as models/prints or provisioning users.

Steps to mitigate the problem:

The first thing to check is the Pyramid summaries panel.

Once on the page, in the bottom right corner of the screen, there is the Tasks panel, which shows the current status of all the tasks in the system.
If many of them are pending, this will be the reason for the above issue.

It is likely that a user sets off a large number of prints and is overloading the system, or there is a need to add more task engines to your system to cope with the load.

To cancel the pending tasks, go to the admin console->Schedules->Task Manager. The Task Manager provides complete visibility on what tasks are running.
A running or pending task can be stopped by enabling the checkbox next to it, theמ stop it with the stop button:

Before you stop the tasks, you must stop the schedule first to prevent it from spanning more tasks.
To find the schedule, you can click on the "Go to Item location" button to see the schedules for that item and stop it.

Microsoft SSAS has a limitation that the "where" clause of the MDX statement can't accept more than one element from the same hierarchy. So for example,  there is no natural way to set a query to show "sales by product where year is 2008 OR year is 2009".
To overcome this there are three options.

1. Aggregation:
   Aggregate the multi-selection of the where statement into a Custom Member. This is the default behavior of Pyramid. If you look at the MDX, you will see something of the following vain.

   MEMBER [date].[year].[combox] as Aggregate([date].[year].[2008], [date].[year].[2009])

   and then see:

   where ([date].[year].[combox])

   The benefit of this option is that the multi-select CAN include custom members and custom sets. However, there are scenarios where this query either takes a VERY long time to return or never returns at all.

2. List:
   It is also possible to supply a list into the "where" clause.
   
   {[date].[year].[2008],[date].[year].[2009]}
   
   However, that might not work well if there is a very large list of items or if specialized logic or aggregations are needed.
    
3. Sub-Query:
   The "From" clause of the MDX statement looks like this:
   
   FROM ( SELECT StripCalculatedMembers({[date].[year].[2008], [date].[year].[2009]}) ON 0 FROM [Pyramid Sales Demo] )
   
   This approach can sometimes perform better than the first approach but has the downside of not being able to include custom or even calculated members (created in Pyramid or created in the SSAS engine respectively). For this reason, the Aggregate option is the default.
   
   Aside from perhaps performance, and that the subquery mode doesn't allow for custom elements, there should be no difference in the results.

Unfortunately, when working against SSAS the cube designs and data configurations are vast, and it's never "one size fits all" when querying it.

For more details on enabled subquery mode see online help here.

To prevent services from over-consuming the resources, Pyramid has a setting that limits the amount of memory that each service can consume.

In some cases, Pyramid Support might request memory and thread dumps to analyze the performances of the services in addition to the Pyramid System/Transaction logs and the windows event server logs.

If the services with high memory are the Runtime/Task engines or the Router, we will need to analyze the Java Process and Thread dumps:

Thread dump of Open JDK Service:

To generate a thread dump, you will need to open the task manager and add the 'Command-Line' column.
There you can determine which service is consuming the memory and whether it's exceeding the limit:

After you've located the required service, open the command line with administrative rights, and run the following command:

"C:\Program Files\Pyramid\Java\Bin\jcmd" 25528 Thread.print > C:\Temp\ThreadDump_RTR.txt

*Replace 25528 with the PID of the java process with high memory usage
**Please note that the path to the jcmd may be slightly different depending on your environment, so just change the path as needed by looking at the folder structure.

Then send us the "ThreadDump_RTR.txt" file created on the c drive us.

Process dump of Open JDK Service:

For the same service, run this command:

"C:\Program Files\Pyramid\java\bin\jcmd" 25528 GC.heap_dump c:\heapDump.hprof

*Replace 25528with the PID of the java process with high memory usage
**Please note that the path to the jcmd may be slightly different depending on your environment, so just change the path as needed by looking at the folder structure.

Then send us the "heapDump" file created on the c drive to us.

If the service that consumes the high memory is the DNC, you will need to do the following:

Memory Dump for Windows Service:

right-click on the DNC/Pyramid Windows Connector and choose "create dump file"
Open the file location and attach send the dump file to us.

We found that the cause was browsers extensions.
As it turns out, Google had changed the chrome extension API, which caused some extensions to break.
These broken extensions throw errors to the console and cause overall slowness.

To resolve,
Turn off all extensions (chrome://extensions/), 
Then close and open the browser

To check if the extensions you need throws errors, turn each on, and click F12 for the Chrome dev tools, then choose the 'Console' tab (if it's not already open on that tab.
 

If you continue to experience performance issues, contact our support team.