Many forms of malicious activity on the Internet involve opening a website in a browser. In phishing, for instance, a malicious page holds the form which the victim has to fill in. Unwanted contents can open from redirects in a page, etc. Hence, the analysis of users' browsing activity can be an important measure to decrease security risks. 

In the present blog, we demonstrate how one can analyze web browsing history with the TIP Malware Check API to find out if a page in a suspicious domain has been visited. We assume a Windows 10 environment with Microsoft Edge, the default browser; a rather common setting. We intend this as an inspiration to administrators of Windows networks in office environments: the script presented here can be easily modified so that it can be scheduled to run regularly, check all users' browsing history and send notifications if something suspicious has been found. 

It is also not hard to modify it to work with any other browser. We will implement it in PowerShell, so it will be a native Windows solution. We use the legacy PowerShell 5.2 for the demonstration as it is there in any Windows 10 system at the moment. 

The basic idea behind the script is as follows. Edge stores the browsing history in an SQLite file at a standard location; for the user "JohnDoe" it is typically here:

C:\Users\JohnDoe\AppData\Local\Microsoft\Edge\User Data\Default\History

What we shall do is the following: we read all URLs from this history, deduce all the domain names from it, and run these domain names through the TIP Malware API. In the present version, we will just report the result verbosely. 

There are many ways in PowerShell to access SQLite databases; we opt for a very basic but lightweight solution that installs easily and is sufficient for our goals. To install it, we start a PowerShell window as administrator and run

Install-Module PSSQLite

(If it does not work, a bit of a tweak is needed for TLS compatibility reasons: you may need to run

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

beforehand, c.f., for instance, this blog. Also, the system might claim that this is from an untrusted source but it is safe to accept and install it.

Another prerequisite is the API Key to use the TIP Malware API. Getting the API key requires a free subscription, while extensive use of the API requires some credits to be purchased. The API key is a string which we will refer to as "YOUR_API_KEY" in what follows. 

Having all ingredients at hand, let's jump directly into the code. Our CMDlet reads: 

[CmdletBinding()]
param([string]$nDays="7")
 
#Check and customize if needed: 
$EdgeHistoryDB = $HOME +'\AppData\Local\Microsoft\Edge\User Data\Default\History'
#End of customization
 
$BaseUrl = "https://api.threatintelligenceplatform.com/v1/malwareCheck?apiKey=" + $ENV:APIKEY +"&domainName="
 
$Query = "SELECT url from urls where CAST(JulianDay(DATETIME())-JulianDay(datetime(last_visit_time / 1000000 + (strftime('%s', '1601-01-01')), 'unixepoch', 'localtime')) as Integer) <= $nDays"
Invoke-SqliteQuery -DataSource $EdgeHistoryDB -Query $query -As PSObject | ForEach-Object {
$Domain = $_.url.Split('/')[2]
$Domain 
} | Sort -Unique | ForEach-Object {
$DomainName = $_
$URI = $BaseUrl + $DomainName
try{
$APIResponse = Invoke-WebRequest -Uri $URI -UseBasicParsing
$MalwareCheckResult = ConvertFrom-Json $APIResponse.Content
$DomainData = [PSCustomObject]@{
DomainName = $DomainName
safeScore = $MalwareCheckResult.safeScore
warningDetails = $(Out-String -Width 2048 -InputObject $MalwareCheckResult.warningDetails) -replace "`n|`r"}
}
catch{
Write-Host "Ran into an issue: $($PSItem.ToString())"
$DomainData = [PSCustomObject]@{
DomainName = $DomainName
safeScore = 0.0
warningDetails = "ERROR"}
}
if($DomainData.safeScore -eq 100.0) {
$textcolor = "DarkGreen"}else{
$textcolor ="Red"}
 
Write-Host -ForegroundColor $textcolor $DomainData

The script has a single command-line option $nDays, which specifies how many days to go back in the browser history. The default is 7 days to avoid analyzing too lengthy histories. (If the script is run regularly, it is best to set it to the running period. 

The location of the Edge history file is stored in $EdgeHistoryDB and it is set to the default location for the current user. It may need some customization. 

We keep the base URL of the API in $BaseUrl, which refers to the API key which we get from the environment variable $ENV:APIKEY The $Query variable holds the SQL line to get the URLs in variables. The query is a bit complex; we leave the filtering according to the difference of the last visit time and the current date in days to SQLite. 

The Invoke-SqliteQuery call will read the URLs from the history file. (We remark here that the access to the history could have been done in a more elaborate manner, see e.g. this code which covers other browsers, too.) 

The result of the history query is processed in a pipe so that each URL is it is split to fields, and the third element (n.b. the numbering starts with 0) is returned. So, for instance, from https://threatintelligenceplatform.com/domain-malware-check-api we get threatintelligenceplatform.com. 

The so-generated stream of domain names are sorted and filtered for unique domains. These are processed in the next pipeline. For each domain name we invoke the API with Invoke-WebRequest and read out the structured result with ConvertFrom-Json. (Further details of this can be learned from this blog). We do exception handling here to avoid stopping because of possibly ill-formed domain names. 

Finally, we simply print out the structured query result with Write-Host. The color of this printout, however, will depend on the result as before the Write-Host we branch on the safeScore returned by the API. If it is equal to 100, the domain hasn't been found in any blacklist, therefore we write it in green. If, however, another score is obtained, the domain is suspicious, so it will be printed in red. 

Before running the script, you need to store the API key in $ENV:APIKEY by doing

$ENV:APIKEY="YOUR_API_KEY"

replacing of course "YOUR_API_KEY" with your actual key. Running the script, the result will look like this:

So in the present form, the CMDlet can be used to analyze a single user's history, which is reasonable even for end-users. Certainly, admins can develop this little code much further, for instance: 

• A loop can be organized to go through all the users' histories. 
• Instead of writing the result, the code can be modified to send notifications to the user and the administrator if something suspicious was found in the history. 
• It can be used on a regular basis to detect visits to suspicious pages. 

Using the TIP Malware Check API in such an automated way from native Windows tools can help a lot to maintain browsing security.