Even though there is significant research activity aimed at the automated detection of malicious domains, blacklists are still the primary information sources to find out if an Internet domain is suspicious for any malicious activities. The comprehensive search for a particular domain can be complicated as there are a great many useful blacklists. Threat Intelligence Platform's interactive dashboard includes an online search for the investigated domain in a fine selection of relevant blacklist. But what if one has a large list of domains to investigate? The Malware Check functionality is also provided by TIP through the RESTFul Domain Malware Check API. In the present blog, we illustrate how to make an automated malware lookup of many domains by using a Windows 10 workstation and its native tools only. We will accomplish the task with a PowerShell script. 

To understand what we are doing exactly, we’ll need lower intermediate programming skills. The code itself can be used, however, even without a detailed understanding. Our demonstration will use the legacy PowerShell 5.2 and PowerShell ISE as editors to enter the code. All of this is available as a standard part of a Windows 10 installation. (Certainly, one can go for newer PowerShell versions along with Visual Studio on the basis of the following description, and it should also work on other platforms such as MacOS X or Linux.) 

Our scenario is as follows: We are given a list of domains in a text file, one domain per line, like this:

bitcovid-3x.cc
threatintelligenceplatform.com
microsoft.com

Certainly, the list can be much longer but we will use this one as an illustration. We want to convert this file to a comma-separated values (csv) file having the information returned by the domain malware check API. Then we will be able to open this file in our favorite spreadsheet software and see which of the domains are to be found in various blacklists. 

The Domain Malware Check API handles GET requests involving an API key and a domain name. Getting the API key requires a free subscription, while extensive use of the API requires some credits be purchased. The API key is a string which we will refer to as "YOUR_API_KEY" in what follows: substitute this with your API key in the code. 

Returning to our goal, what we obviously need to do is the following: Looping through the domain list: 

• make the respective GET request to the API;
• decode the result, add it to the domain name and export it as a csv line.

This can be implemented in PowerShell with the following code:

#Extends a list of domain names in a text file to a csv with TIP Malware Check results
# The input list does not contain a header and contains valid domains
#Usage:
# 1. Replace YOUR_API_KEY with your working API key
# 2. Once done, you can run it like this:
#   PS C:\Users\User\WorkingDirectory> .\ExtendDomainCsvWithMalwareCheck.ps1 '.\inputfile.csv' '.\outputfile.csv'
# Note: if the output file exists and is not empty, the results will be appended.
 
[CmdletBinding()]
param(
[string] $InputFile,
[string] $OutputFile
)
 
$APIKey = "YOUR_API_KEY"
 
$BaseUrl = "https://api.threatintelligenceplatform.com/v1/malwareCheck?apiKey=" + $APIKEY +"&domainName="
 
Import-Csv -Header "domain" -Path $InputFile | ForEach-Object {
Write-Host $_.domain
$URI = $BaseUrl + $_.domain
$Domain = $_.domain
try{
$APIResponse = Invoke-WebRequest -Uri $URI -UseBasicParsing
Write-Host $APIResponse
$MalwareCheckResult = ConvertFrom-Json $APIResponse.Content
$DomainData = [PSCustomObject]@{
DomainName = $_.domain
safeScore = $MalwareCheckResult.safeScore
warningDetails = $(Out-String -Width 2048 -InputObject $MalwareCheckResult.warningDetails) -replace "`n|`r"}
}
catch{
$DomainData = [PSCustomObject]@{
DomainName = $Domain
safeScore = 0.0
warningDetails = "ERROR"}
}
$DomainData | Export-Csv -Append -NoTypeInformation -Encoding UTF8 $OutputFile
  }

Save this under a reasonable name, say, ExtendDomainCsvWithMalwareCheck.ps1, with PowerShell ISE or your favorite editor, and replace "YOURAPIKEY" with your API key. Before actually using this Cmdlet, let's see what is under the hood. 

The first few lines hold comments with a description of what it is and how to invoke. The param block describes the two positional arguments: the names of the input and output files. (It is not the most elegant way of having command-line arguments in a Cmdlet, but now we want to keep things simple. Then we define two variables: $APIKey holds the API key, whereas $BaseUrl is the beginning of the API URL so that we just have to add the domain to complete it. (See also the API docs.) 

Now the main part of the script comes. Remember that we have a text file with one domain in each line; a simple csv file after all, so we use the Import-Csv Cmdlet to read the file. The -Header option describes the name of the single column ("domain") as we have no header line, while -Path specifies the actual csv file. The read lines, which are thus parsed to be represented with objects are pipelined to the ForEach-Object, to loop through each line. 

In the kernel of the loop we first echo the actual domain name, $_.domain with Write-Host: when running, we will see in the command window what is actually going on. Then we put together the actual query string of the API into the $URI variable: We concatenate the $BaseUrl and the actual domain name. 

Next, we store the domain name itself in the variable $Domain. This is needed because the loop variable $_.domain$ is not visible in the catch branch. But why do we use exception handling here? Assume that our input file eventually contains some bad lines like invalid domain names. Certainly, we don't want our script to stop, and we also want to get informed about this. Hence, we try calling the API with Invoke-WebRequest. The -UseBasicParsing option is there to avoid any dependency from the legacy Internet Explorer. We store the response in $APIResponse, and echo it to keep track of what is going on. Normally, the API will return a JSON structure that is parsed to an object by ConvertFrom-JSON. The response JSON will be like this:

{
 "safeScore":40,
 "warningDetails":[
   "Status: dangerous. Reported by Quttera: malicious site"
 ]
}

To the pipeline we send a PSCustomObject whose attributes will then get exported as the fields of the actual line of the output csv file. This is put together from the API response. As for the warning details, it is a JSON array but we are not interested in its internal structure. Therefore, we convert it to a string with Out-String, and remove the newline (-replace) which is there by default in the output of this Cmdlet. (Note that this latter can be treated more elegantly with the -NoNewline option in higher versions of PowerShell, but we want to keep our code compatible with series 5 PowerShell.) 

But what if something goes wrong? Invoke-WebRequest will drop an exception which we catch. In the catch branch, we set up a PSCustomObject of the same structure as normally, we keep the domain name (which is probably wrong), we give a safe score of 0.0, which never occurs normally, and put "ERROR" into the warning details field. 

Having set up $DomainData this way, we append it to the csv file whose name comes from the $OutputFile$ parameters using the Export-Csv Cmdlet. We use the -NoTypeInformaiton option as we don't want to have any supplemental information in the file, just the header and the records. (Note that this Cmdlet has more options in newer versions of PowerShell. For instance, the safeScore field will be quoted, as the -UseQoutes option appears from PowerShell version 7 on.) Having understood our code, let's give it a try. Assuming that the domain list is in inputfile.csv in the same directory as the script itself (C:\Users\User\WorkingDirectory in the example below), as written in the comment we do the following in the PowerShell prompt:

PS C:\Users\User\WorkingDirectory> .\ExtendDomainCsvWithMalwareCheck.ps1 '.\inputfile.csv' '.\outputfile.csv'

 (If you have not used PowerShell scripts yet, you may need to enable scripting on your system, which is disabled by default. It is the ExecutionPolicy that controls this behavior; check e.g. this page, especially if your system complains that ps1's cannot be loaded because scripting is disabled on your system.) As the script is running, the following appears in the console window:

while the output file, outputfile.csv will have the desired contents:

"DomainName","safeScore","warningDetails"
"bitcovid-3x.cc","86,36","Listed on Virus Total suspicious URLs analyser"
"threatintelligenceplatform.com","100",""
"microsoft.com","100",""

just as desired. So, we have the csv file extended with the malware analysis results, and the same could have been done with thousands of domains. The csv can be imported to Excel by a simple double click, and the investigation results can be further studied. Do not hesitate to visit https://threatintelligenceplatform.com to try all of these yourself and find other useful threat intelligence tools.