Troubleshooting

Cyware Fusion and Threat Response

Troubleshooting

This topic provides some basic commands that are useful for troubleshooting issues during and after the deployment:

Manage Docker Swarm

Following are some basic Docker Swarm commands that are useful for troubleshooting issues:

To check the Docker Swarm Node status, run the following command. This command lists all the nodes of the Docker Swarm Manager.

docker node ls

Sample Output

ID                           HOSTNAME        STATUS  AVAILABILITY  MANAGER STATUS
41bcef6utixb0l0ca7gxuivsj0    swarm-worker2   Ready   Active
538ciaotwjuritcdtn9npbnkuz    swarm-worker1   Ready   Active
6e216jshn25ckzbvmwlnh5jr3g *  swarm-manager1  Ready   Active        Leader

Note

The status for each node should be Ready. If the above command times out or throws any error, then you must set up the swarm cluster again.

To list all Docker services, run the following command. This command lists services are running in the swarm. The REPLICAS column shows both the actual and desired number of tasks for the service. If the service is in replicated-job or global-job, it will additionally show the completion status of the job as completed tasks over total tasks the job will execute.
```
docker service ls
```
Sample Output
```
ID            NAME      MODE            REPLICAS             IMAGE
4c8wgl7q4ndfd  frontend  replicated      5/5                  nginx:alpine
5dmu1ept4cxcf  redis     replicated      3/3                  redis:3.0.6
6hh08h9uu8uwr  job       replicated-job  1/1 (3/5 completed)  nginx:late
```
To check if containers are up and running, run the following command:
```
docker ps
```
If the actual and desired number of replicas for a service is not the same, run the following command to check for the cause:
```
docker service ps --no-trunc <service-name>
```
To check the Docker Overlay Network info, run the following command:
```
docker network ls
```
To check if the Docker systemd service is running, run the following command:
```
systemctl status docker
```
To check the Docker systemd logs, run the following command:
```
journalctl -u -f docker.service
```

To view Docker service-wise stats on the current server, run the following command:
```
docker stats
```
To restart the Docker service on the server, run the following command. This command restarts all services that run as Docker containers.
```
sudo systemctl restart docker
```
Note
You must manually restart the services that do not run as Docker containers.
To update a service, run the following command:
```
docker service update <service_name> --force
```
To delete a stack, run the following command:
```
docker stack rm <stack_name>
```

To go inside the containers, run the following command:

docker exec -it $(docker ls -q- -f name=<service_name>) bash

To go inside a specific container, run the following command:
```
docker exec -it <container_id> bash
```

To redeploy a stack, run the following command:

docker stack deploy -c <compose-file-name> <stack_name>

View Application Logs

Following are some commands to view the application logs of various Respond services:

To view the backend application logs, run the following command:

cd /apps/cyware/logs/cftr_backend/application
tail -f <logfile.name>

To view the Celery beat logs, run the following command:
```
cd /apps/cyware/logs/celery-beat
tail -f <logfile.name>
```
To view the Celery notification logs, run the following command:
```
cd /apps/cyware/logs/celery-notification
tail -f <logfile.name>
```
To view the Celery worker logs, run the following command:
```
cd /apps/cyware/logs/celery-worker
tail -f <logfile.name>
```

To view the frontend logs, run the following command:

cd /apps/cyware/logs/cftr_frontend
tail -f <logfile.name>

View Datastore Logs (Docker Version)

Following are some commands to view the logs of the database services that are deployed as Docker containers.

To view the Elasticsearch logs, run the following command:
```
docker service logs -f db_elasticsearch
```
To view the Redis logs, run the following command:
```
docker service logs -f db_redis
```
To view the PostgreSQL logs, run the following command:
```
docker service logs -f db_postgres
```
To view the Object Storage logs, run the following command:
```
docker service logs -f db_object_storage
```

View Datastore Logs (Systemd Version)

Following are some commands to view the logs of the database services that are not deployed as Docker containers.

To view the PostgreSQL logs, run the following command:
```
postgres
```
To view the Redis logs, run the following command:
```
tail -f /var/log/redis/*.log
```
To view the Elasticsearch logs, run the following command:
```
tail -f /
```

Check Network Connectivity

Following are some commands to check the network connectivity.

To check the connectivity on a TCP port, run the following command:
```
telnet <HOSTNAME/IP> <PORT>
```
To check the connectivity on a UDP port, run the following command:
```
nc -vz -u <HOSTNAME/IP> <PORT>
```

Manage Proxy Settings

Following are some commands to manage proxy settings.

To view the current proxy settings, run the following command:
```
printenv|grep proxy
```

To add proxy settings to the current session, run the following command:

export http_proxy=<PROXY>
export https_proxy=<PROXY>
export no_proxy=<IPs,HOSTNAMES>

To add proxy settings for a user, open the ~/.bashrc file and add the following lines:
```
export http_proxy=<PROXY>
export https_proxy=<PROXY>
export no_proxy=<IPs,HOSTNAMES>
```
Note
You must sign in again to apply the proxy settings.
To add global proxy settings, open the /etc/environment file and add the following lines:
```
http_proxy=<PROXY>
https_proxy=<PROXY>
no_proxy=<IPs,HOSTNAMES>
```
To add proxy settings for the Docker systemd service, open the /usr/lib/systemd/system/docker.service file and add the following lines:
```
[Service]
Environment="HTTP_PROXY=<PROXY>"
Environment="HTTPS_PROXY=<PROXY>"
```
You must restart Docker to apply the proxy settings. To restart the Docker services, run the following command:
```
systemctl restart docker
```

General Troubleshooting Commands

Following are some commands to view the system uptime and system-level logs.

To view the uptime of the system, run the following command:
```
uptime
```
Sample Output
```
9:49  up 3 days, 18:56, 2 users, load averages: 2.34 2.97 4.65
```
To view the system-level logs for reboot and more, run the following command:
```
tail -f /var/log/messages
```

View Nginx Logs

If Nginx is installed, run the following command to view the Nginx reverse proxy logs -

tail -f /var/log/nginx/access.log
tail -f /var/log/nginx/error.log

Was this helpful?

Would you like to provide feedback? Just click here to suggest edits.

Cyware Fusion and Threat Response

Troubleshooting

Manage Docker Swarm

Note

Note

View Application Logs

View Datastore Logs (Docker Version)

View Datastore Logs (Systemd Version)

Check Network Connectivity

Manage Proxy Settings

Note

General Troubleshooting Commands

View Nginx Logs

Search results