Import Intel into Intel Exchange
Note
The Bulk File Upload feature for Import Intel is available in Intel Exchange v3.7.5.0 (EA) onwards.
You can import intel from structured data sources like STIX bundles, CSV files, and OpenIOC into Intel Exchange and use these imported sources to create intel within the platform. The following table shows the supported intel sources and the respective supported file formats to import:
Before you start
Ensure that you have Create Intel permission.
To import intel from a file that may include invalid data, ensure that importing partially valid files is enabled in Administration > Configuration > General Settings > Import Intel Preference. If partial import is disabled, then the import fails if the STIX file includes invalid data. For more information, see Configure General Settings.
Steps
To import intel into Intel Exchange, follow these steps:
Click +New and select Import Intel. A note on the form indicates whether partial import is enabled or disabled.
Drag and drop one or more files into the upload area, or click Browse to select files from your system.
The selected files appear in a list under the Files section. Each file entry has a corresponding Collection dropdown next to it.
For each file, select the Collection to store the imported data. By default, the collection is set to the Default collection. You can choose a different collection as needed. Collection is a grouping of data, and the imported data is stored as part of that collection. You can later use the collection name to run rules for the imported intel. To create a new Import Collection, see Create Import Collection.
If only one file is selected, a preview of the file is displayed before import.
Click Import.
Note
You can upload up to 10 files per import action. Each file must be 10 MB or smaller, and the supported formats are .json, .stix, .xml, or .csv.
Objects of the file are parsed and ingested into the platform in the background. You can view the import status in Intel History. The following section describes the status of an import:
Pending: The ingestion is still pending.
Processing: The ingestion is in process.
Created: Ingested successfully and created threat data objects.
Failed: Failed to ingest.
Partially successful: If the file data consists of invalid data or missing data, the platform will discard this data and set the status to Partially Successful. You can click on Download Logs to download the error log in CSV format.
Manage Import Intel History
You can view the details of the import in Intel History, such as the imported file name, email ID of the importer, import date, and import status. You can click the vertical ellipses of an import and select the following activities in the Intel History:
View: Displays the list of threat data objects ingested from the file.
Export to CSV: Downloads the list of ingested objects in CSV format.
To view the details of the partially created or failed intel creations, click Download Logs. A CSV file is downloaded that includes error details for the failed objects.
Import Cyware CSV Format
Intel Exchange provides a custom CSV file format to map the file data with the components that Intel Exchange support. You can manually enter the intel gathered from various sources and create a custom CSV file, or you can also modify the CSV file to map with Intel Exchange CSV format. Moreover, you can import this file into the platform and create intel. The Cyware CSV template includes the following column headers:
Malware | Attack Pattern | Course of Action | Campaign |
Indicator-ipv4 | Indicator-ipv6 | Indicator-URL | Indicator-Email Address |
Identity | Indicator-Domain | Indicator-SHA1 | Indicator-SHA224 |
Indicator-SHA256 | Indicator-SHA384 | Indicator-SHA512 | Indicator-MD5 |
Indicator-SSDEEP | Infrastructure | Intrusion Set | Location* |
Tags | Threat Actor | Tool | Vulnerability |
Description | TLP | Confidence | External References |
*For more information about headers, see ???.
Before you start
Ensure that you have Create Intel permission.
To import intel from a file that may include invalid data, ensure that importing partially valid files is enabled in Administration > Configuration > General Settings > Import Intel Preference. If partial import is disabled, then the import fails if the file includes invalid data. For more information, see Configure General Settings.
Steps
To import intel using the Cyware CSV file format, follow these steps:
Click +New in the upper right corner and select Import Intel.
Click Download CSV Template to download the template to your local system. You can use this template to enter the intel data to be imported.
Note
The platform processes the first 50,000 records from the imported file, including empty rows.
Drag and drop one or more CSV files into the upload area, or click Browse to select files from your system.
The selected files appear in a list under the Files section. Each file entry has a corresponding Collection dropdown next to it.
For each file, select the Collection to store the imported data. By default, the collection is set to the Default collection. You can choose a different collection as needed. Collection is a grouping of data, and the imported data is stored as part of that collection. You can later use the collection name to run rules for the imported intel. To create a new Import Collection, see Create Import Collection.
If only one file is selected, a preview of the file is displayed before import. You can modify the headers, and if the uploaded file includes invalid headers, the column headers are highlighted in red. Click the invalid header and select a valid header from the drop-down.
If multiple files are selected and any file contains invalid headers, the import action fails after you click Import. The affected file is marked in red. You must correct the headers and re-upload the file to proceed with the import.
Click Import.
Note
You can upload up to 10 files per import action. Each file must be 10 MB or smaller, and the supported formats are .json, .stix, .xml, or .csv.
Objects of the file are parsed and ingested into the platform in the background. You can view the import status in Intel History. You will receive an in-app notification when the intel is created.
After the import is complete, a report object is created. The objects in the first column of the imported file are called primary objects and are ingested as related objects of the report object. Objects in subsequent columns of the imported file are called secondary objects and are ingested as related objects of the primary objects.
Cyware (CSV): Use Case and Interpretation
The following table is an example of an imported CSV (Cyware) file:
Malware | Indicator-ipv4 | Domain |
|---|---|---|
emotet | 1.1.1.1 | sampledomain1.com |
heodo | 2.2.2.2 | sampledomain2.com |
The following are a few points to consider while importing a CSV (Cyware) file:
The first row in the CSV displays the STIX Domain Object names and metadata, such as Malware, Attack Pattern, Campaign, Tags, Descriptions, and more. If there is a spelling mistake or a mismatch in the names, such as malware being written as malware, the CSV file is not processed.
If the data is incorrect, such as an IP address 1.1.1.1 is written as 1.1.a.1. the file is still processed for all the correct data and only the incorrect data is not processed. Overall the CSV file is partially processed. You can download error logs in a CSV file.
A report object is created with the filename.csv. The objects in the first column are directly related to the created report object as part of ingestion. For example, from the provided table, the malware emotet and heodo are related to the report object with the filename.csv. Relationships are formed between the object in the first column and the others in the same row. For example, from the provided table, the malware emotet is related to indicator 1.1.1.1 and sampledomain1.com, while heodo malware is related to indicator 2.2.2.2 and sampledomain2.com.
When you add a TLP, confidence score, and tags to an object of a record, the same information is automatically attached to all its related objects. The confidence score is attached as a Source Confidence for all objects. You can add a maximum of three tags to an object.
You can add a description and attach a maximum of three external references in the form of URLs to the first object in the record.
Use ',' as a separator for different values in a record. For example,
<indicator_value>,<malware_value>,<attack_pattern>, <TLP>, <tag>,....
Use '|' as a separator for different values in a single column of a record. You can use | as a separator for tags and external references. Additionally, you can only add three tags or external references while importing intel. For example,
<tag1|tag2|tag3>, <external_reference1|external_reference2|external_reference3>
Use a break or a new line as a separator among records. For example,
<indicator_value>, <malware_value>, <TLP>, <tag>,... <indicator_value>, <attack_pattern>, <TLP>, <tag1|tag2>,.... <malware_value>, <indicator_value1|indicator_value2>, <tag>,...