Skip to main content

General Documents

  • General Documents
  • Generic Articles
  • Configure Cyware Advance Threat Intel Crawler v3.0

Configure Cyware Advance Threat Intel Crawler v3.0

Note

This feature is available in Intel Exchange v3.7.5.0 (EA) onwards.

Cyware Advance Threat Intel Crawler v3.0 is a web browser extension that enables the extraction of STIX domain objects and the creation of intel from parsed information. The browser extension automatically scans and detects threat intel from any web-based content, such as emails, blogs, threat bulletins, or any raw feeds. Cyware Advance Threat Intel Crawler v3.0 employs advanced Artificial Intelligence models to enhance effectiveness, precision, and coverage across a wide range of domain objects.

When you sign in to the Intel Exchange platform, the browser extension validates the active session and enables you to ingest the threat intel extracted from content into the platform. It can further categorize the intel into threat data objects, such as indicators, malware, threat actors, attack patterns, and more.

Cyware offers the Cyware Advance Threat Intel Crawler v3.0 browser extension for Intel Exchange platform. The Cyware Advance Threat Intel Crawler v3.0 supports all valid HTML-based website URLs, and CSV, TXT, HTML, and PDF file types to scan and detect IOCs. 

Before you Start 

  • You must have access to the Intel Exchange platform.

  • You must have full permission to the Intel Exchange platforms.

    Note

    For on-premise deployments:

    • Ensure you have enabled the installation and execution of the browser extension directly within the browser.

    • Add the URLs to your Allow List that are required to ensure bi-directional connectivity. Contact Cyware support for the list of URLs.

Cyware Advance Threat Intel Crawler v3.0 In Intel Exchange

Cyware Advance Threat Intel Crawler v3.0 integrates with Intel Exchange to offer the following features:

  • Operationalize threat intelligence: Bulk extract IOCs and other domain objects from web pages and HTTPS-based PDF files and ingest them to Intel Exchange. The threat intel created using Cyware Advance Threat Intel Crawler v3.0 is directly added to Intel Exchange and is immediately available for review and publishing. Analysts can update the intel to add more context and publish it as needed. 

  • Threat Intel Lookup: Obtain a quick summary of the intel by hovering over the highlighted entities on the web page. Use Intel Exchange to extend your research and gain more context on the threat data. You can filter data created using Cyware Advance Threat Intel Crawler v3.0 using the Source value as a Browser Extension in Threat Data.

  • Export Threat Intel: Export the threat intelligence data into a CSV file for reporting, collaboration, or offline analysis.

  • Multi-Tenancy and Automatic login: Use Cyware Advance Threat Intel Crawler v3.0 with multiple instances of the Intel Exchange platform. To get started, sign in to Intel Exchange and launch the Cyware Advance Threat Intel Crawler v3.0 from the Intel Exchange application's browser tab.

  • Multi-Browser support: Cyware Advance Threat Intel Crawler v3.0 is compatible with Chrome, Edge, and other Chromium-based browsers.

Install Cyware Advance Threat Intel Crawler v3.0 in Chrome Browser

Install the appropriate version of the extension in your Chrome browser based on your Intel Exchange version to scan and extract threat data objects from web-based content. 

Note

Supported Cyware Threat Intel Crawler Versions 

Customers must install the new plugin (version 3.0) to start using it with Intel Exchange 3.7.5.0 and above. There is no upgrade path from version 2.0.1 to 3.0.

Steps 

To install and configure the Cyware Advance Threat Intel Crawler v3.0 in Google Chrome or any Chromium-based browser, follow these steps:

  1. Open Google Chrome and go to Chrome Web Store. 

  2. Sign in to Chrome Web Store and search for Cyware Advance Threat Intel Crawler v3.0 extension. 

  3. Select the appropriate version and click Add to Chrome.

  4. In the confirmation pop-up, click Add Extension to grant the required permissions.

  5. To allow the extension to access local file URLs:

    1. Go to Extensions > Manage Extensions.

    2. Click Details on Cyware Advance Threat Intel Crawler v3.0 extension.

    3. Turn on Allow access to file URLs.

You can now use Cyware Advance Threat Intel Crawler v3.0 to scan web-based content and extract threat data objects on the Chrome browser.

Install Cyware Advance Threat Intel Crawler v3.0 Extension in Edge Browser

To install and configure Cyware Advance Threat Intel Crawler v3.0 on the Microsoft Edge browser, follow these steps:

  1. Open Microsoft Edge, and go to Chrome Web Store.

  2. Search for Cyware Advance Threat Intel Crawler v3.0 and select the extension from the list.

  3. In the top banner, click Get extension.

  4. Click Add Extension on the pop-up to allow the platform access to the extension.

  5. To enable file URL access:

    1. Go to Extensions > Manage Extensions.

    2. Click Details on Cyware Advance Threat Intel Crawler v3.0.

    3. Turn on Allow access to file URLs.

You can now use Cyware Advance Threat Intel Crawler v3.0 to scan web-based content and extract threat data objects on the Microsoft Edge browser.

Scan Content Using Cyware Advance Threat Intel Crawler v3.0

To scan content and extract threat data objects using Cyware Advance Threat Intel Crawler v3.0, follow these steps:

  1. Sign in to the Intel Exchange application.

  2. Go to Extensions and select Cyware Advance Threat Intel Crawler v3.0. You can view the name of the signed-in Intel Exchange instance from the extension header and log out as needed.

  3. Click Scan Page. The extension scans the current webpage or web-hosted PDF and extracts structured threat data objects (SDOs).

  4. Enter the Report Title for the report.

  5. Review the extracted SDOs, displayed under two tabs:

    • New SDOs: Displays SDOs identified on the page that are not yet present in Intel Exchange.

      • Use the search bar to locate specific objects.

      • Click the + button beside the search bar to add a new object.

      • Click the + icon next to a category to add additional values of that SDO type.

      • Use the checkboxes to select or deselect SDOs for inclusion in the report.

      • Click on ellipsis beside an SDO to edit or delete.

    • Matched SDOs: Displays SDOs that already exist in Intel Exchange.

      • By default, no matched SDOs are selected.

      • Use the checkboxes to select or deselect SDOs for inclusion in the report.

      • Click on a matched SDO to view key details such as confidence score, TLP marking, and created date.

      Note

      Matched SDOs cannot be edited or deleted.

  6. To add additional details to the intel, click Add Metadata and enter the following details:

    • Description: Enter a description for the intel.

    • TLP: Select a TLP for the intel.

    • Confidence score: Select a confidence score for the intel.

    • Deprecates after: Set the duration in days after which the threat data (indicator) will be deprecated, unless the source provides an expiry duration (Range: 1–180 days). If the same indicator is received from multiple sources, the longest valid duration is applied.

    • Tags: Add tags to categorize the intel.

    • Custom Scores: Enter the values for the custom scores configured by the administrator in Administration > Configuration > Custom Scores.

    • External Reference: Auto-filled with the scanned page URL. You can review and edit this field to reference the original source of the intel.

    • Apply Metadata to all Objects: Select this option to apply the metadata to all selected objects of the intel. If you do not select this option, then the metadata is applied only to the report object created for the extension.

      Note

      The description is added only to the report object and not to the objects you selected to include in the intel.

  7. (Optional) Click Download CSV to export the selected SDOs in CSV format.

  8. Click Proceed to review your selections and continue to the intel creation step.

  9. Select the Intel Exchange instance(s) where the report should be created. The logged-in instance is selected by default.

  10. Click Create Intel.

  11. (Optional) To add more objects to a specific type, or if no data is found:

    1. Click Add Object for an object type.

    2. Enter the object value and press Enter.

  12. Click Create Intel. After creation, you can access the intel from the Threat Data module for further analysis or action.

In Intel Exchange, intel is created with the following default parameters:

  • Source: Browser Extension in the STIX source category.

  • Collection: Same as the browser used to scan. For example, for the Chrome browser, the default collection is Google Chrome.

Usage Limits

The following limits and parsing behaviors apply when scanning PDF files using Cyware Advance Threat Intel Crawler v3.0.

  • You can scan PDF files using the extension in Chrome and Edge browsers only.

  • Cyware Advance Threat Intel Crawler v3.0 parses a maximum of 1 MB of data.

  • Cyware Advance Threat Intel Crawler v3.0 scans for a maximum of 10,000 IOCs per scan.

  • IOC values, such as URLs, domains, hashes, and more, that extend beyond a single line are truncated.

  • Parsing of data with unbalanced parentheses, square brackets, or curly brackets is not supported.