Import Intel into Intel Exchange
You can import intel from structured data sources like STIX bundles, CSV files, and OpenIOC into Intel Exchange and use these imported sources to create intel within the platform. The following table shows the supported intel sources and the respective supported file formats to import:
Intel Source | File Format |
|---|---|
STIX 1.x | XML |
STIX 2.0 | JSON |
STIX 2.1 | JSON |
STIX 1.x URL | URL |
MISP | JSON |
CSV (Cyware) | CSV |
CSV (Recorded Future) | CSV |
OpenIOC | XML |
Before you start
Ensure that you have Create Intel permission.
To import intel from a file that may include invalid data, ensure that importing partially valid files is enabled in Administration > Configuration > General Settings > Import Intel Preference. If partial import is disabled, then the import fails if the file includes invalid data. For more information, see Configure General Settings.
Steps
To import intel into Intel Exchange, follow these steps:
Click +New and select Import Intel. A note under the drop-down lists indicates if partial import is enabled or disabled.
Enter the following details:
Select Format: Select the format of the file to be imported. For example, STIX 2.0.
Collection: Select a collection to store the imported data. By default, the collection of the selected format is automatically selected. You can select a different collection to store the imported data. Collection is a grouping of data and the imported data is stored as a part of the collection. You can later use the collection name to run rules for the imported intel.
Click Upload File to browse and upload a file of size less than or equal to 10 MB.
Click Import.
Objects of the file are parsed and ingested into the platform in the background. You can view the import status in Intel History. The following section describes the status of an import:
Pending: The ingestion is still pending.
Processing: The ingestion is in process.
Created: Ingested successfully and created threat data objects.
Failed: Failed to ingest.
Partially successful: If the file data consists of invalid data or missing data the platform will discard these data and set the status to Partially Successful. You can click on Download Logs to download the error log in CSV format.
What are the default collections of the import file formats?
The following table shows the default collections of the import file formats.
File Format | Default Collection |
|---|---|
STIX 1.x | xml |
STIX 2.0 | Stix2 |
STIX 2.1 | Stix2 |
STIX 1.x URL | url |
MISP | misp |
CSV (Cyware) | csv |
CSV (Recorded Future) | csv |
Open IOC | openioc |
Manage Import Intel History
You can view the details of the import in Intel History, such as the imported file name, email ID of the importer, import date, and import status. You can click the vertical ellipses of an import and select the following activities in the Intel History:
View: Displays the list of threat data objects ingested from the file.
Export to CSV: Downloads the list of ingested objects in CSV format.
To view the details of the partially created or failed intel creations, click Download Logs. A CSV file is downloaded that includes error details for the failed objects.
Import Cyware CSV Format
Intel Exchange provides a custom CSV file format to map the file data with the components that Intel Exchange supports. You can manually enter the intel gathered from various sources and create a custom CSV file or you can also modify the CSV file to map with Intel Exchange CSV format. Moreover, you can import this file into the platform and create intel. The Cyware CSV template includes the following column headers:
Malware | Attack Pattern | Course of Action | Campaign |
Indicator-ipv4 | Indicator-ipv6 | Indicator-URL | Indicator-Email Address |
Identity | Indicator-Domain | Indicator-SHA1 | Indicator-SHA224 |
Indicator-SHA256 | Indicator-SHA384 | Indicator-SHA512 | Indicator-MD5 |
Indicator-SSDEEP | Infrastructure | Intrusion Set | Location* |
Tags | Threat Actor | Tool | Vulnerability |
Description | TLP | Confidence | External References |
*For more information about headers, see Header Information.
Before you start
Ensure that you have Create Intel permission.
To import intel from a file that may include invalid data, ensure that importing partially valid files is enabled in Administration > Configuration > General Settings > Import Intel Preference. If partial import is disabled, then the import fails if the file includes invalid data. For more information, see Configure General Settings.
Steps
To import intel using the Cyware CSV file format, follow these steps:
Click +New in the upper right corner and select Import Intel.
Select the import file format as CSV (Cyware) from Select Format.
Click Download Template in Select Format to download the template to your local system. You can use this template to enter the intel data to be imported.
Note
The platform processes the first 10,000 records from the imported file, including empty rows.
Select a collection to store the imported data. By default, the
csvcollection is automatically selected. You can select a different collection to store the imported data. Collection is a grouping of data and the imported data is stored as a part of the collection. You can later use the collection name to run rules for the imported intel.Click Upload File to browse and upload a file within 10MB in size. You can also upload a CSV file and modify the header. If the uploaded file includes invalid column headers, the column headers are highlighted in red.
Click on the invalid header and select a valid header from the drop-down.
Note
Column data with invalid headers are not ingested into the platform.
Click Import.
Objects of the file are parsed and ingested into the platform in the background. You can view the import status in Intel History. You will receive an in-app notification when the intel is created.
After the import is complete, a report object is created. The objects in the first column of the imported file are called primary objects and are ingested as related objects of the report object. Objects in subsequent columns of the imported file are called secondary objects and are ingested as related objects of the primary objects.
The location header can include the following allowed values:
'Afghanistan', 'Albania', 'Algeria', 'American Samoa', 'Andorra', 'Angola', 'Anguilla', 'Antarctica', 'Antigua and Barbuda', 'Argentina', 'Armenia', 'Aruba', 'Australia', 'Austria', 'Azerbaijan', 'Bahamas', 'Bahrain', 'Bangladesh', 'Barbados', 'Belarus', 'Belgium', 'Belize', 'Benin', 'Bermuda', 'Bhutan', 'Bolivia, Plurinational State of', 'Bonaire, Sint Eustatius and Saba', 'Bosnia and Herzegovina', 'Botswana', 'Bouvet Island', 'Brazil', 'British Indian Ocean Territory', 'Brunei Darussalam', 'Bulgaria', 'Burkina Faso', 'Burundi', 'Cambodia', 'Cameroon', 'Canada', 'Cape Verde', 'Cayman Islands', 'Central African Republic', 'Chad', 'Chile', 'China', 'Christmas Island', 'Cocos (Keeling) Islands', 'Colombia', 'Comoros', 'Congo', 'Congo, the Democratic Republic of the', 'Cook Islands', 'Costa Rica', 'Croatia', 'Cuba', 'Curaçao', 'Cyprus', 'Czech Republic', "Côte d'Ivoire", 'Denmark', 'Djibouti', 'Dominica', 'Dominican Republic', 'Ecuador', 'Egypt', 'El Salvador', 'Equatorial Guinea', 'Eritrea', 'Estonia', 'Ethiopia', 'Falkland Islands (Malvinas)', 'Faroe Islands', 'Fiji', 'Finland', 'France', 'French Guiana', 'French Polynesia', 'French Southern Territories', 'Gabon', 'Gambia', 'Georgia', 'Germany', 'Ghana', 'Gibraltar', 'Greece', 'Greenland', 'Grenada', 'Guadeloupe', 'Guam', 'Guatemala', 'Guernsey', 'Guinea', 'Guinea-Bissau', 'Guyana', 'Haiti', 'Heard Island and McDonald Islands', 'Holy See (Vatican City State)', 'Honduras', 'Hong Kong', 'Hungary', 'Iceland', 'India', 'Indonesia', 'Iran, Islamic Republic of', 'Iraq', 'Ireland', 'Isle of Man', 'Israel', 'Italy', 'Jamaica', 'Japan', 'Jersey', 'Jordan', 'Kazakhstan', 'Kenya', 'Kiribati', "Korea, Democratic People's Republic of", 'Korea, Republic of', 'Kuwait', 'Kyrgyzstan', "Lao People's Democratic Republic", 'Latvia', 'Lebanon', 'Lesotho', 'Liberia', 'Libya', 'Liechtenstein', 'Lithuania', 'Luxembourg', 'Macao', 'Macedonia, the Former Yugoslav Republic of', 'Madagascar', 'Malawi', 'Malaysia', 'Maldives', 'Mali', 'Malta', 'Marshall Islands', 'Martinique', 'Mauritania', 'Mauritius', 'Mayotte', 'Mexico', 'Micronesia, Federated States of', 'Moldova, Republic of', 'Monaco', 'Mongolia', 'Montenegro', 'Montserrat', 'Morocco', 'Mozambique', 'Myanmar', 'Namibia', 'Nauru', 'Nepal', 'Netherlands', 'New Caledonia', 'New Zealand', 'Nicaragua', 'Niger', 'Nigeria', 'Niue', 'Norfolk Island', 'Northern Mariana Islands', 'Norway', 'Oman', 'Pakistan', 'Palau', 'Palestine, State of', 'Panama', 'Papua New Guinea', 'Paraguay', 'Peru', 'Philippines', 'Pitcairn', 'Poland', 'Portugal', 'Puerto Rico', 'Qatar', 'Romania', 'Russian Federation', 'Rwanda', 'Réunion', 'Saint Barthélemy', 'Saint Helena, Ascension and Tristan da Cunha', 'Saint Kitts and Nevis', 'Saint Lucia', 'Saint Martin (French part)', 'Saint Pierre and Miquelon', 'Saint Vincent and the Grenadines', 'Samoa', 'San Marino', 'Sao Tome and Principe', 'Saudi Arabia', 'Senegal', 'Serbia', 'Seychelles', 'Sierra Leone', 'Singapore', 'Sint Maarten (Dutch part)', 'Slovakia', 'Slovenia', 'Solomon Islands', 'Somalia', 'South Africa', 'South Georgia and the South Sandwich Islands', 'South Sudan', 'Spain', 'Sri Lanka', 'Sudan', 'Suriname', 'Svalbard and Jan Mayen', 'Swaziland', 'Sweden', 'Switzerland', 'Syrian Arab Republic', 'Taiwan, Province of China', 'Tajikistan', 'Tanzania, United Republic of', 'Thailand', 'Timor-Leste', 'Togo', 'Tokelau', 'Tonga', 'Trinidad and Tobago', 'Tunisia', 'Turkey', 'Turkmenistan', 'Turks and Caicos Islands', 'Tuvalu', 'Uganda', 'Ukraine', 'United Arab Emirates', 'United Kingdom', 'United States', 'United States Minor Outlying Islands', 'Uruguay', 'Uzbekistan', 'Vanuatu', 'Venezuela, Bolivarian Republic of', 'Viet Nam', 'Virgin Islands, British', 'Virgin Islands, U.S.', 'Wallis and Futuna', 'Western Sahara', 'Yemen', 'Zambia', 'Zimbabwe', and 'Åland Islands'.
Cyware (CSV): Use Case and Interpretation
The following table is an example of an imported CSV (Cyware) file:
Malware | Indicator-ipv4 | Domain |
|---|---|---|
emotet | 1.1.1.1 | sampledomain1.com |
heodo | 2.2.2.2 | sampledomain2.com |
The following are a few points to consider while importing a CSV (Cyware) file:
The first row in the CSV displays the STIX Domain Object names and metadata, such as Malware, Attack Pattern, Campaign, Tags, Descriptions, and more. If there is a spelling mistake or a mismatch in the names such as malware being written as malwre, the CSV file is not processed.
If the data is incorrect, such as an IP address 1.1.1.1 is written as 1.1.a.1. the file is still processed for all the correct data and only the incorrect data is not processed. Overall the CSV file is partially processed. You can download error logs in a CSV file.
A report object is created with the filename.csv. The objects in the first column are directly related to the created report object as part of ingestion. For example, from the provided table, the malware emotet and heodo are related to the report object with filename.csv. Relationships are formed between the object in the first column and the others in the same row. For example, from the provided table, the malware emotet is related to indicator 1.1.1.1 and sampledomain1.com, while heodo malware is related to indicator 2.2.2.2 and sampledomain2.com.
When you add a TLP, confidence score, and tags to an object of a record, the same information is automatically attached to all its related objects. The confidence score is attached as a Source Confidence for all objects. You can add a maximum of three tags to an object.
You can add a description and attach a maximum of three external references in the form of URLs to the first object in the record.
Use ',' as a separator for different values in a record. For example,
<indicator_value>,<malware_value>,<attack_pattern>, <TLP>, <tag>,....
Use '|' as a separator for different values in a single column of a record. You can use | as a separator for tags and external references. Additionally, you can only add three tags or external references while importing intel. For example,
<tag1|tag2|tag3>, <external_reference1|external_reference2|external_reference3>
Use break or new line as a separator among records. For example,
<indicator_value>, <malware_value>, <TLP>, <tag>,... <indicator_value>, <attack_pattern>, <TLP>, <tag1|tag2>,.... <malware_value>, <indicator_value1|indicator_value2>, <tag>,...