Skip to main content

General Documents

Configure Cyware Threat Intel Crawler

Cyware Threat Intel Crawler is a web browser extension that enables the extraction of STIX domain objects and the creation of intel from parsed information. The browser extension automatically scans and detects threat intel from any web-based content, such as emails, blogs, threat bulletins, or any raw feeds. Cyware Threat Intel Crawler employs advanced Artificial Intelligence models to enhance effectiveness, precision, and coverage across a wide range of domain objects.

When you sign in to the Intel Exchange or Collaborate Member Portal platform, the browser extension validates the active session and enables you to ingest the threat intel extracted from content into the platforms. It can further categorize the intel into threat data objects, such as indicators, malware, threat actors, attack patterns, and more.

Cyware offers the Cyware Threat Intel Crawler browser extension for CTIX and CSAP products. The Cyware Threat Intel Crawler supports all valid HTML-based website URLs, and CSV, TXT, HTML, and PDF file types to scan and detect IOCs. 

Before you start 

  • You must have access to the CTIX or CSAP Member Portal platforms as per your requirement.

  • You must have full permission to the Collaborate Member Portal and Intel Exchange platforms.

Note

For on-premise deployments:

  • Ensure you have enabled the installation and execution of the browser extension directly within the browser.

  • Add the URLs to your Allow List that are required to ensure bi-directional connectivity. Contact Cyware support for the list of URLs.

Cyware Threat Intel Crawler in CTIX

Cyware Threat Intel Crawler integrates with Intel Exchange to offer the following features:

  • Operationalize threat intelligence: Bulk extract IOCs from web pages and HTTPS-based PDF files and ingest them to CTIX. Later analysts can update and publish the threat intel as required. The intel created using Cyware Threat Intel Crawler is saved in draft mode and you can edit it to add more context and publish it per your requirements.

  • Threat Intel Lookup: Obtain a quick summary of the intel by hovering over the highlighted entities on the web page. Use Intel Exchange to extend your research and gain more context into the threat data. You can filter data created using Cyware Threat Intel Crawler using the Source value as Browser Extension in Threat Data.

  • Export Threat Intel: Export the threat intelligence data into a CSV file for reporting, collaboration, or offline analysis.

  • Multi-Tenancy and Automatic login: Use Cyware Threat Intel Crawler with multiple instances of the CTIX platform. To get started, sign in to CTIX and launch the Cyware Threat Intel Crawler from the Intel Exchange application's browser tab.

  • Multi-Browser support: Cyware Threat Intel Crawler is compatible with Chrome, Firefox, Edge, and other Chromium-based browsers.

Cyware Threat Intel Crawler in CSAP

Cyware Threat Intel Crawler collaborates with Collaborate to offer the following features:

  • Create and share threat intel based on the information parsed by Cyware Threat Intel Crawler. The intel created using Cyware Threat Intel Crawler is saved in draft mode and you can edit it to add more context and publish it per your requirements.

  • View the created intel from Share Threat Intel in the Member Portal. Cyware Threat Intel Crawler functionality is limited to users of the Member Portal.

  • Collect and leverage threat intel directly from the web, without manually copying and pasting data. This can save time and improve responsiveness and information sharing.

  • Export the threat intelligence data into a CSV file for reporting, collaboration, or offline analysis.

Install Cyware Threat Intel Crawler Extension in Chrome Browser

Install the appropriate version of the Cyware Threat Intel Crawler extension in your Chrome browser based on your Intel Exchange and Cyware Collaborate versions to scan and extract IOCs from web-based content.

Supported Cyware Threat Intel Crawler Versions 

Cyware Threat Intel Crawler Version 

Cyware Product Version

2.0

Intel Exchange version 3.5.x.x and later

1.1.3

Intel Exchange versions before 3.5.x.x

Cyware Collaborate all versions

Steps 

To install and configure Cyware Threat Intel Crawler in the Chrome browser, follow these steps:

  1. Open Google Chrome, and go to Google Web Store.

  2. Sign in to Chrome Web Store, and search for Cyware Threat Intel Crawler.

  3. Select the appropriate version of Cyware Threat Intel Crawler, and click Add to Chrome.

  4. Click Add Extension on the pop-up to allow platform access to the extension. 

  5. To enable Cyware Threat Intel Crawler to access file URLs:

    1. Go to Extensions > Manage Extensions.

    2. Click Details on Cyware Threat Intel Crawler extension.

    3. Turn on Allow access to file URLs.

You can now use Cyware Threat Intel Crawler to scan web-based content and extract IOCs on the Chrome browser.

Install Cyware Threat Intel Crawler Extension in Mozilla Firefox Browser

To install Cyware Threat Intel Crawler from Mozilla Add-On, follow the steps:

  1. Open Mozilla Firefox, and go to Mozilla Add-On.

  2. Search for Cyware Threat Intel Crawler and select the extension from the list.

  3. Click Add to Firefox.

  4. Click Add on the pop-up to allow platform access to the extension.

You can now use Cyware Threat Intel Crawler to scan web-based content and extract IOCs on the Mozilla Firefox browser.

Install Cyware Threat Intel Crawler Extension in Edge Browser

To install and configure Cyware Threat Intel Crawler on the Microsoft Edge browser, follow these steps:

  1. Open Microsoft Edge browser, and go to Chrome Web Store.

  2. Search and select the Cyware Threat Intel Crawler extension.

  3. In the top banner, click Get extension.

    Crawler_Edge.png

  4. Click Add Extension on the pop-up to allow platform access to the extension. 

  5. To enable Cyware Threat Intel Crawler to access file URLs:

    1. Go to Extensions > Manage Extensions.

    2. Click Details on Cyware Threat Intel Crawler extension.

    3. Turn on Allow access to file URLs.

You can now use Cyware Threat Intel Crawler to scan web-based content and extract IOCs on the Microsoft Edge browser.

Scan Content Using Cyware Threat Intel Crawler

To scan content and extract IOCs using Cyware Threat Intel Crawler, follow these steps:

  1. Sign in to the Intel Exchange or Collaborate application.

  2. Open the URL of the website or PDF file you want to scan. You can scan web-hosted PDF files only.

  3. Go to Extensions and select Cyware Threat Intel Crawler. You can view the name of the Intel Exchange or Cyware Collaborate instance you are signed in.

  4. Click Scan Now. Cyware Threat Intel Crawler scans, parses threat data, and categorizes the identified threat data objects into various types.

    You can download the IOCs in CSV format for offline analysis

  5. (Optional) To add more objects to a specific type, follow these steps:

    1. Click Add Object for an object type.

    2. Enter the object value and press Enter.

  6. Click Create Intel.

  7. Select the Intel Exchange and Collaborate instances to create intel.

  8. Click Create Intel.

In Collaborate, a draft intel is created in the Member Portal. In Intel Exchange, intel is created with the following default parameters:

  • Source: Browser Extention in the STIX source category.

  • Collection: Same as the browser used to scan. For example, for the Chrome browser, the default collection is Google Chrome.

Scanning PDF Files

Remember the following points before scanning PDF files:

  • You can scan PDF files in the Chrome and Firefox browsers only.

  • Cyware Threat Intel Crawler parses a maximum of 1 MB of data.

  • Cyware Threat Intel Crawler scans for a maximum of 100,000 characters.

  • IOC values, such as URLs, domains, and more, that spill over to the next line are partially parsed.

  • Parsing of data with unbalanced parentheses, square brackets, or curly brackets is not supported.