Increased Startup Time for Ask Data Elasticsearch Causes Enable to Time Out

Issue

Ask Data's Elastic search engine takes a long time to start up (that is change from red to yellow state). The following information is true based on your Tableau Server version:

For Tableau Server 2019.1.0 and version 2019.1.1, the elasticserver service (Elasticsearch) takes longer than 90 seconds to start up. This causes enable to fail, blocks Ask Data (the nlp service), and blocks Tableau Server from restarting successfully.
From Tableau Server 2019.1.2 and later 2019.1.x versions, the elasticserver service (Elasticsearch) takes longer than 300 seconds to start up. This causes enable to fail, blocks Ask Data (the nlp service), and blocks Tableau Server from restarting successfully.
For Tableau Server 2019.2 and later versions, the elasticserver service (Elasticsearch) takes longer than 300 seconds to start up. This blocks Ask Data (the nlp service): user queries cannot be processed.

Environment

Tableau Server 2019.1 - 2019.4

Resolution

Option 1

Upgrade to Tableau Server 2020.1 or a newer version. Click here for downloads of current and previous versions of Tableau Server: Tableau Server Downloads and Release Notes. For more information on current releases, see Upgrade Tableau Server and Server Upgrade.

Option 2 - Only for Tableau Server 2019.1.0 or 2019.1.1

If you are unable to upgrade, as a temporary workaround, a Server Admin for the Tableau Server host may try the following steps:

Using the TSM Admin UI, try to identify which host is running the Elasticsearch service. There should only be one.
Locate the services/elasticserver_0.near.0.0.0/manifest.json file.
In that file, edit the following lines:
- - Set agentEnableTimeoutSeconds to 400
- - Set enableTimeoutSeconds to 300
- Save the manifest.json file
Find the Elasticserver process (java.exe on Windows, java on Linux) and inspect to make sure you have the right Java process.
Kill the Elasticserver Java process, and let the system bring it back up.
If the symptoms persist, increase the values and try again.

Option 3 - For Tableau Server 2019.1.2 - 2019.4

If you are unable to upgrade, as a temporary workaround, a Server Admin for the Tableau Server host may try the following steps:

For Tableau Server on Linux

Stop Elasticserver
- First stop tabadminagent: sudo su -l tableau -c "systemctl --user stop tabadminagent_0.service"
- Then call /var/opt/tableau/tableau_server/data/tabsvc/services/elasticserver*/disable.sh for the Elasticserver service.
Go to the host with the elasticserver service. On Tableau 2019.2 or higher, you may have multiple hosts; if so, do steps 3-5 on each
Navigate to /var/opt/tableau/tableau_server/data/tabsvc/elasticserver/0/nodes/0
Move the directory indices/ to a different name (within the same directory is fine).
Enable the Elasticserver service:
- /var/opt/tableau/tableau_server/data/tabsvc/services/elasticserver*/enable.sh/
- /var/opt/tableau/tableau_server/data/tabsvc/services/elasticserver*/data/run-sgadmin.sh
Start tabadminagent sudo su -l tableau -c "systemctl --user start tabadminagent_0.service"
Go to the server’s TSM Admin Web GUI, check if all NLP services are ACTIVE and HEALTHY.
- If any of the nlp services does not have status ACTIVE and HEALTHY, go to the corresponding node and identify the service: ps aux | grep nlp . Kill the process using the process id found. Then tabsvc will automatically restart the terminated NLP service.

For Tableau Server on Windows

Stop Elasticserver
- First stop tabadminagent: net stop tabadminagent_0
- Then call C:\ProgramData\Tableau\Tableau Server\data\tabsvc\services\elasticserver*\disable.cmd for the Elasticserver service.
Go to the host with the elasticserver service. On Tableau 2019.2 or higher, you may have multiple hosts; if so, do steps 3-5 on each
Navigate to C:\ProgramData\Tableau\Tableau Server\data\tabsvc\elasticserver\0\nodes\0
Rename the indices/ folder to a different name (within the same directory is fine).
Enable the Elasticserver service:
- C:\ProgramData\Tableau\Tableau Server\data\tabsvc\services\elasticserver*\enable.cmd
- C:\ProgramData\Tableau\Tableau Server\data\tabsvc\services\elasticserver*\data\run-sgadmin.cmd
Start tabadminagent net start tabadminagent_0
Go to the server’s TSM Admin Web GUI, check if all NLP services are ACTIVE and HEALTHY.
- If any of the nlp services does not have status ACTIVE and HEALTHY, go to the corresponding node and kill the service in the Process Manager. Then tabsvc will automatically restart the terminated NLP service.

Cause

In Tableau Server 2020.1, changes were made to the architecture of Elasticsearch which prevent this issue from occurring.

Additional Information

The issue occurs during server restarts. A server admin/operator might realize that not all the services have come up. Both the NLP and Elasticsearch (also called Elasticserver) services could be unhealthy in the TSM Admin View or with tsm status -v, and the restart command would not have succeeded.

For more information about Ask Data and Elastic Server, see Ask Data and Elastic Server Processes.

Discuss this article... Feedback Forum