Versions Compared

Version Old Version 11 New Version Current
Changes made by

Adrian Nedelcu

Anjali Tyagi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

When troubleshooting various problems on the Content Gateway one might need to capture a Java thread dump. This KB article explains how to do that using jstack.

Prerequisites

Note

CentOS/RHEL 7 is EOL and yum commands on CentOS 7 now fail with "Could not retrieve mirrorlist http://mirrorlist.centos.org/". There is a workaround, but please plan a migration to Rocky Linux/RHEL 8.

jstack needs to be installed, and may not be installed by default. To check whether jstack is installed or not please run the following command:

Code Block
if jstack -h 2>/dev/null; then echo "jstack is Installed"; else echo "jstack is Not Installed"; fi

...

Code Block
yum install java-11-openjdk-devel

Instructions

  1. Find Collect out the PID of the Content Gateway process by running this command

    Code Block
    PID=$(ps aux | grep [c]loudgateway.pid | awk '{ print $2 }')

  2. Capture the thread dump using jstack and the open file descriptors using lsof by running these commands using the PID determined at step 1

    Code Block
    jstack $PID > thread_dump.txt
lsof -p $PID > lsof.txt

  3. If the above command does not capture anything, try running this to force a thread dump from a non-responsive process:

    Code Block
    jstack -F $PID > thread_dump.txt

  4. If you are using JDK 11+, forcing a thread dump requires this command

    Code Block
    jhsdb jstack --locks --pid $PID > thread_dump.txt

  5. Repeat the jstack/lsof commands 3 times total a few seconds apart like this:

    Code Block
    for i in $(seq 3); do jstack $PID > thread_dump-$i.txt; lsof -p $PID > lsof-$i.txt; sleep 60;done

    or ( for JDK 8 )

    Code Block
    for i in $(seq 3); do jstack -F $PID > thread_dump-$i.txt; lsof -p $PID > lsof-$i.txt; sleep 60;done

...