Troubleshooting
This topic provides some basic commands that are useful for troubleshooting issues during and after the deployment:
Manage Docker Swarm
Following are some basic Docker Swarm commands that are useful for troubleshooting issues:
- To check the Docker Swarm Node status, run the following command. This command lists all the nodes of the Docker Swarm Manager. - docker node ls - Sample Output - ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 41bcef6utixb0l0ca7gxuivsj0 swarm-worker2 Ready Active 538ciaotwjuritcdtn9npbnkuz swarm-worker1 Ready Active 6e216jshn25ckzbvmwlnh5jr3g * swarm-manager1 Ready Active Leader - Note- The status for each node should be Ready. If the above command times out or throws any error, then you must set up the swarm cluster again. 
- To list all Docker services, run the following command. This command lists services are running in the swarm. The - REPLICAScolumn shows both the actual and desired number of tasks for the service. If the service is in- replicated-jobor- global-job, it will additionally show the completion status of the job as completed tasks over total tasks the job will execute.- docker service ls - Sample Output - ID NAME MODE REPLICAS IMAGE 4c8wgl7q4ndfd frontend replicated 5/5 nginx:alpine 5dmu1ept4cxcf redis replicated 3/3 redis:3.0.6 6hh08h9uu8uwr job replicated-job 1/1 (3/5 completed) nginx:late 
- To check if containers are up and running, run the following command: - docker ps 
- If the actual and desired number of replicas for a service is not the same, run the following command to check for the cause: - docker service ps --no-trunc <service-name> 
- To check the Docker Overlay Network info, run the following command: - docker network ls 
- To check if the Docker systemd service is running, run the following command: - systemctl status docker 
- To check the Docker systemd logs, run the following command: - journalctl -u -f docker.service 
- To view Docker service-wise stats on the current server, run the following command: - docker stats 
- To restart the Docker service on the server, run the following command. This command restarts all services that run as Docker containers. - sudo systemctl restart docker - Note- You must manually restart the services that do not run as Docker containers. 
- To update a service, run the following command: - docker service update <service_name> --force 
- To delete a stack, run the following command: - docker stack rm <stack_name> 
- To go inside the containers, run the following command: - docker exec -it $(docker ls -q- -f name=<service_name>) bash 
- To go inside a specific container, run the following command: - docker exec -it <container_id> bash 
- To redeploy a stack, run the following command: - docker stack deploy -c <compose-file-name> <stack_name> 
View Application Logs
Following are some commands to view the application logs of various Respond services:
- To view the backend application logs, run the following command: - cd /apps/cyware/logs/cftr_backend/application tail -f <logfile.name> 
- To view the Celery beat logs, run the following command: - cd /apps/cyware/logs/celery-beat tail -f <logfile.name> 
- To view the Celery notification logs, run the following command: - cd /apps/cyware/logs/celery-notification tail -f <logfile.name> 
- To view the Celery worker logs, run the following command: - cd /apps/cyware/logs/celery-worker tail -f <logfile.name> 
- To view the frontend logs, run the following command: - cd /apps/cyware/logs/cftr_frontend tail -f <logfile.name> 
View Datastore Logs (Docker Version)
Following are some commands to view the logs of the database services that are deployed as Docker containers.
- To view the Elasticsearch logs, run the following command: - docker service logs -f db_elasticsearch 
- To view the Redis logs, run the following command: - docker service logs -f db_redis 
- To view the PostgreSQL logs, run the following command: - docker service logs -f db_postgres 
- To view the Object Storage logs, run the following command: - docker service logs -f db_object_storage 
View Datastore Logs (Systemd Version)
Following are some commands to view the logs of the database services that are not deployed as Docker containers.
- To view the PostgreSQL logs, run the following command: - postgres
- To view the Redis logs, run the following command: - tail -f /var/log/redis/*.log 
- To view the Elasticsearch logs, run the following command: - tail -f / 
Check Network Connectivity
Following are some commands to check the network connectivity.
- To check the connectivity on a TCP port, run the following command: - telnet <HOSTNAME/IP> <PORT> 
- To check the connectivity on a UDP port, run the following command: - nc -vz -u <HOSTNAME/IP> <PORT> 
Manage Proxy Settings
Following are some commands to manage proxy settings.
- To view the current proxy settings, run the following command: - printenv|grep proxy 
- To add proxy settings to the current session, run the following command: - export http_proxy=<PROXY> export https_proxy=<PROXY> export no_proxy=<IPs,HOSTNAMES> 
- To add proxy settings for a user, open the - ~/.bashrcfile and add the following lines:- export http_proxy=<PROXY> export https_proxy=<PROXY> export no_proxy=<IPs,HOSTNAMES> - Note- You must sign in again to apply the proxy settings. 
- To add global proxy settings, open the - /etc/environmentfile and add the following lines:- http_proxy=<PROXY> https_proxy=<PROXY> no_proxy=<IPs,HOSTNAMES> 
- To add proxy settings for the Docker systemd service, open the - /usr/lib/systemd/system/docker.servicefile and add the following lines:- [Service] Environment="HTTP_PROXY=<PROXY>" Environment="HTTPS_PROXY=<PROXY>" - You must restart Docker to apply the proxy settings. To restart the Docker services, run the following command: - systemctl restart docker 
General Troubleshooting Commands
Following are some commands to view the system uptime and system-level logs.
- To view the uptime of the system, run the following command: - uptime - Sample Output - 9:49 up 3 days, 18:56, 2 users, load averages: 2.34 2.97 4.65 
- To view the system-level logs for reboot and more, run the following command: - tail -f /var/log/messages 
View Nginx Logs
If Nginx is installed, run the following command to view the Nginx reverse proxy logs -
tail -f /var/log/nginx/access.log tail -f /var/log/nginx/error.log