KNOWLEDGE BASE

After Running Out Of Hard Disk Space A Few Processes Are Not Able To Start


Published: 26 Aug 2020
Last Modified Date: 28 Aug 2020

Issue

After running out of disk space then freeing up space or expanding more disk space, Tableau Server continues to show as DEGRADED and a few processes (like Gateway, Data Engine, Elastic Server, etc) are in an error state.

In ​servicestate.json, the "currentState" of those error state processes may display as "INSTALL_FAILED". By default, the servicestate.json can be found in /var/opt/tableau/tableau_server/data/tabsvc/tabadminagent/0/servicestate.json

    "hyper_0.20202.20.0425.1206" : {
      "currentState" : "INSTALL_FAILED",
      "requestedState" : "ENABLED"
    },

    "elasticserver_0.20202.20.0425.1206" : {
      "currentState" : "INSTALL_FAILED",
      "requestedState" : "ENABLED"
    },

    "gateway_0.20202.20.0425.1206" : {
      "currentState" : "INSTALL_FAILED",
      "requestedState" : "ENABLED"
    },

Environment

  • Linux
  • Tableau Server 2020.2

Resolution

Option 1:

Try to modify the current state of error processes in servicestate.json

  1. Stop Tableau Server (If the error processes were not able to stop successfully ignore it, we will perform an OS reboot)
    • tsm stop
  2. Stop TSM services by script
    • sudo /opt/tableau/tableau_server/packages/scripts.<Build number>/stop-administrative-services
  3. Edit the servicestate.json manually, make sure the current state of all processes is "currentState" : "ENABLED". By default servicestate.json locates in: /var/opt/tableau/tableau_server/data/tabsvc/tabadminagent/0/servicestate.json
  4. Reboot OS
  5. Check if TSM services started automatically
    • tsm status -v
  6. If TSM services didn't start automatically, Start TSM services by script.
    • sudo /opt/tableau/tableau_server/packages/scripts.<Build number>/start-administrative-services
  7. Start Tableau Server
    • tsm start
  8. Check if the status of Tableau Server is "RUNNING"
    • tsm status -v
  9. If a few processes still in error state, try to restart Tableau Server one or two additional times
    • tsm restart
 

Option 2:

Perform a clean install then restore the data in the same machine
  1. Creating the backup and gathering any assets that require manual actions. See Backing up Tableau Server for recovery
  2. Copy the backup files (.tsbak, .json) to a non-Tableau Server related directory
  3. Completely remove Tableau Server without removing server licensing. See To completely remove Tableau Server without removing server licensing
  4. Install the latest maintenance release of the same version. Find the latest maintenance release from Product Downloads and Release Notes
  5. Restoring core Tableau Server functionality and Restore other functionality. See Perform a Full Backup and Restore of Tableau Server

Cause

After running out of disk space and some of the processes turn to degraded status, in some cases, Tableau Server may identify the status of processes incorrectly even after the disk issue resolved and generating the wrong servicestate.json file.

Additional Information

For preventing this issue, please monitor free disk space and enable email notifications about low disk space. For more information, see Configure Server Event Notification
Did this article resolve the issue?