Using the Google Safe Browsing API from Delphi

July 11, 2011 by Rodrigo 6 Comments

The Google Safe Browsing API is a service that enables applications to check URLs against the Google’s lists of suspected phishing and malware pages. Exist two types of APIs for using the Safe Browsing service, Safe Browsing API v2 and Safe Browsing Lookup API in this article I will show how use the Safe Browsing Lookup API from a Delphi application.

The Safe Browsing Lookup API is designed to provide a simple interface through HTTP GET or POST request and get the state of the URL(s) directly from the server.

Like most of the services provided by Google you need to request an API key. In order to obtain an API key you must log in with your existing Google account and sign up for the API at http://www.google.com/safebrowsing/key_signup.html

Using the GET Method

The Get method allow to the client only lookup one URL per request. To use the GET method you must make a request to this URL

https://sb-ssl.google.com/safebrowsing/api/lookup?client=CLIENT&apikey=APIKEY&appver=APPVER&pver=PVER&url=URL

Parameters

The client parameter indicates the type of client, it could be any name of the client’s choice.
The apikey parameter indicates the API key.
The appver parameter indicates the version of the client.
The pver parameter indicates the protocol version that the client supports. Currently this should be “3.0”. The format is “major.minor”. If we update the protocol, we will make sure that minor revisions are always compatible; however major revision will be incompatible and the server MAY NOT be able to cope with an older protocol.
The url parameter indicates the url the client wants to lookup. It must be a valid URL (non ASCII characters must be in UTF-8) and needs to be encoded properly to avoid confusion. For example, if the url contains ‘&’, it could be interpreted as the separator of the CGI parameters. We require the API users to use the percent encoding for the set of “reserved characters”, which is defined in RFC 3986 . A summary of the percent encoding can be found here.

Check this Sample Url

https://sb-ssl.google.com/safebrowsing/api/lookup?client=mydemoapp&<strong>apikey</strong>=1234567890&appver=1.0.1&pver=3.0&url=http%3A%2F%2Fwww.google.com%2F

In this case the values passed are
client = mydemoapp apikey = 1234567890 appver = 1.0.1 pver = 3.0 url = http://www.google.com

Response

The service returns the following HTTP response codes for the GET method

200: The queried URL is either phishing, malware or both, see the response body for the specific type.
204: The requested URL is legitimate, no response body returned.
400: Bad Request — The HTTP request was not correctly formed.
401: Not Authorized — The apikey is not authorized
503: Service Unavailable .

Additionally the server will include the actual type of URL in the response body when the queried URL matches either the phishing or malware lists, so the body will contain one of these values
“phishing” | “malware” | “phishing,malware"

Delphi Code for the GET Request

The next source uses the Wininet functions to make the GET request, feel free to use another components like Indy or synapse to accomplish this task.

{$APPTYPE CONSOLE}
uses
  Classes,
  Windows,
  WinInet,
  SysUtils;
const
  sUserAgent = 'Mozilla/5.001 (windows; U; NT4.0; en-US; rv:1.0) Gecko/25250101';
  //¡¡¡¡¡¡¡¡¡¡Please be nice and use your own API key, get a key from here http://code.google.com/apis/safebrowsing/key_signup.html ¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡
  sApiKey    = 'ABQIAAAAzY4CKjsBFYV4Rxx0ZQaKlxQL2a1oqOk9I7UVXAZVtWa6uSA2XA';
  sServer    = 'sb-ssl.google.com';
  sGetSafeBrowsing   = '/safebrowsing/api/lookup?client=delphi&apikey=%s&appver=1.5.2&pver=3.0&url=%s';

//this function translate a WinInet Error Code to a description of the error.
function GetWinInetError(ErrorCode:Cardinal): string;
const
   winetdll = 'wininet.dll';
var
  Len: Integer;
  Buffer: PChar;
begin
  Len := FormatMessage(
  FORMAT_MESSAGE_FROM_HMODULE or FORMAT_MESSAGE_FROM_SYSTEM or
  FORMAT_MESSAGE_ALLOCATE_BUFFER or FORMAT_MESSAGE_IGNORE_INSERTS or  FORMAT_MESSAGE_ARGUMENT_ARRAY,
  Pointer(GetModuleHandle(winetdll)), ErrorCode, 0, @Buffer, SizeOf(Buffer), nil);
  try
    while (Len > 0) and {$IFDEF UNICODE}(CharInSet(Buffer[Len - 1], [#0..#32, '.'])) {$ELSE}(Buffer[Len - 1] in [#0..#32, '.']) {$ENDIF} do Dec(Len);
    SetString(Result, Buffer, Len);
  finally
    LocalFree(HLOCAL(Buffer));
  end;
end;

//make a GET request using the WinInet functions
function Https_Get(const ServerName,Resource : string;Var Response:AnsiString): Integer;
const
  BufferSize=1024*64;
var
  hInet    : HINTERNET;
  hConnect : HINTERNET;
  hRequest : HINTERNET;
  ErrorCode : Integer;
  lpvBuffer : PAnsiChar;
  lpdwBufferLength: DWORD;
  lpdwReserved    : DWORD;
  dwBytesRead     : DWORD;
  lpdwNumberOfBytesAvailable: DWORD;
begin
  Result  :=0;
  Response:='';
  hInet := InternetOpen(PChar(sUserAgent), INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);

  if hInet=nil then
  begin
    ErrorCode:=GetLastError;
    raise Exception.Create(Format('InternetOpen Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
  end;

  try
    hConnect := InternetConnect(hInet, PChar(ServerName), INTERNET_DEFAULT_HTTPS_PORT, nil, nil, INTERNET_SERVICE_HTTP, 0, 0);
    if hConnect=nil then
    begin
      ErrorCode:=GetLastError;
      raise Exception.Create(Format('InternetConnect Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
    end;

    try
      //make the request
      hRequest := HttpOpenRequest(hConnect, 'GET', PChar(Resource), HTTP_VERSION, '', nil, INTERNET_FLAG_SECURE, 0);
      if hRequest=nil then
      begin
        ErrorCode:=GetLastError;
        raise Exception.Create(Format('HttpOpenRequest Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
      end;

      try
        //send the GET request
        if not HttpSendRequest(hRequest, nil, 0, nil, 0) then
        begin
          ErrorCode:=GetLastError;
          raise Exception.Create(Format('HttpSendRequest Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
        end;

          lpdwBufferLength:=SizeOf(Result);
          lpdwReserved    :=0;
          //get the status code
          if not HttpQueryInfo(hRequest, HTTP_QUERY_STATUS_CODE or HTTP_QUERY_FLAG_NUMBER, @Result, lpdwBufferLength, lpdwReserved) then
          begin
            ErrorCode:=GetLastError;
            raise Exception.Create(Format('HttpQueryInfo Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
          end;

         if Result=200 then //read the body response in case which the status code is 200
          if InternetQueryDataAvailable(hRequest, lpdwNumberOfBytesAvailable, 0, 0) then
          begin
            GetMem(lpvBuffer,lpdwBufferLength);
            try
              SetLength(Response,lpdwNumberOfBytesAvailable);
              InternetReadFile(hRequest, @Response[1], lpdwNumberOfBytesAvailable, dwBytesRead);
            finally
              FreeMem(lpvBuffer);
            end;
          end
          else
          begin
            ErrorCode:=GetLastError;
            raise Exception.Create(Format('InternetQueryDataAvailable Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
          end;

      finally
        InternetCloseHandle(hRequest);
      end;
    finally
      InternetCloseHandle(hConnect);
    end;
  finally
    InternetCloseHandle(hInet);
  end;
end;

//encode a Url
function URLEncode(const Url: string): string;
var
  i: Integer;
begin
  Result := '';
  for i := 1 to Length(Url) do
  begin
    case Url[i] of
      'A'..'Z', 'a'..'z', '0'..'9', '-', '_', '.':
        Result := Result + Url[i];
    else
        Result := Result + '%' + IntToHex(Ord(Url[i]), 2);
    end;
  end;
end;

//Send The GET request and process the returned body
Procedure TestGet(const AUrl : string);
var
 Response     : AnsiString;
 ResponseCode : Integer;
begin
   ResponseCode:=Https_Get(sServer,Format(sGetSafeBrowsing,[sApiKey,URLEncode(AUrl)]), Response);
   case ResponseCode of
     200: Writeln(Format('The queried URL (%s) is %s',[AUrl,Response]));
     204: Writeln(Format('The queried URL (%s) is %s',[AUrl,'legitimate']));
     400: Writeln('Bad Request — The HTTP request was not correctly formed.');
     401: Writeln('Not Authorized — The apikey is not authorized');
     503: Writeln('Service Unavailable — The server cannot handle the request.');
   else
         Writeln('Unknow response');
   end;
end;

begin
  try
     //Now check some urls.
     TestGet('http://malware.testing.google.test/testing/malware/');
     TestGet('orgsite.info');
     TestGet('http://www.google.com');
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
  Readln;
end.

This will return

The queried URL (http://malware.testing.google.test/testing/malware/) is malware
The queried URL (orgsite.info) is malware
The queried URL (http://www.google.com) is legitimate

Using the POST Method

The post request is more powerful because the client can also look up a set of URLs (up to 500) through HTTP POST request. To use the POST method you must make a request to this URL

https://sb-ssl.google.com/safebrowsing/api/lookup?client=CLIENT&apikey=APIKEY&appver=APPVER&pver=PVER

Parameters

The client parameter indicates the type of client, it could be any name of the client’s choice.
The apikey parameter indicates the API key.
The appver parameter indicates the version of the client.
The pver parameter indicates the protocol version that the client supports.

Check this Sample Url

https://sb-ssl.google.com/safebrowsing/api/lookup?client=mydemoapp&<strong>apikey</strong>=1234567890&appver=1.0.1&pver=3.0

Request Body

The client specifies the queried URLs in the POST request body using the following format:
POST_REQ_BODY = NUM LF URL (LF URL)* NUM = (DIGIT)+ URL = url string following the RFC 1738
The request’s body contains several lines separated by LF. The first line is a number indicating how many URLs are included in the body. The next several lines are URLs to be looked up. Each line contains one URL and the client must specify at least one URL in the body.

check this sample
2 http://www.google.com/ http://malware.testing.google.test/testing/malware/

Response

The server generates the following HTTP response codes for the POST request:

200: AT LEAST ONE of the queried URLs are matched in either the phishing or malware lists, the actual results are returned through the response body
204: NONE of the queried URLs matched the phishing or malware lists, no response body returned
400: Bad Request — The HTTP request was not correctly formed
401: Not Authorized.
503: Service Unavailable.

Body

In the POST request, the server will return a list of URLs queried in the response body when at least one of the queried URLs matches in the suspected phishing or malware lists.
POST_RESP_BODY = VERDICT (LF VERDICT)* VERDICT = “phishing” | “malware” | “phishing,malware” | “ok”

The type has the same meaning as in the GET response body except that some of the URLs may be legitimate (recall that the server returns empty content only when none of the queried URLs matches the phishing or malware lists). In this case, we return “ok” for the non-matching URLs. The results are separated by the LF. There is a one-on-one mapping between the results in the response body and the queried URLs in the request body. For example, assume there are 10 URLs specified in the request body, the server will return exactly 10 results with the original order. That is, the first line corresponds to the result of the first queried URL, the second line corresponds to the result of the second queried URL, and so on.

Delphi Code for the POST Request

{$APPTYPE CONSOLE}
uses
  Classes,
  Windows,
  WinInet,
  SysUtils;

const
  sUserAgent = 'Mozilla/5.001 (windows; U; NT4.0; en-US; rv:1.0) Gecko/25250101';
  //¡¡¡¡¡¡¡¡¡¡Please be nice and use your own API key, get a key from here http://code.google.com/apis/safebrowsing/key_signup.html ¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡
  sApiKey    = 'ABQIAAAAzY4CKjsBFYV4Rxx0ZQaKlxQL2a1oqOk9I7UVXAZVtWa6uSA2XA';
  sServer    = 'sb-ssl.google.com';
  sPostSafeBrowsing  = '/safebrowsing/api/lookup?client=delphi&apikey=%s&appver=1.5.2&pver=3.0';

function GetWinInetError(ErrorCode:Cardinal): string;
const
   winetdll = 'wininet.dll';
var
  Len: Integer;
  Buffer: PChar;
begin
  Len := FormatMessage(
  FORMAT_MESSAGE_FROM_HMODULE or FORMAT_MESSAGE_FROM_SYSTEM or
  FORMAT_MESSAGE_ALLOCATE_BUFFER or FORMAT_MESSAGE_IGNORE_INSERTS or  FORMAT_MESSAGE_ARGUMENT_ARRAY,
  Pointer(GetModuleHandle(winetdll)), ErrorCode, 0, @Buffer, SizeOf(Buffer), nil);
  try
    while (Len > 0) and {$IFDEF UNICODE}(CharInSet(Buffer[Len - 1], [#0..#32, '.'])) {$ELSE}(Buffer[Len - 1] in [#0..#32, '.']) {$ENDIF} do Dec(Len);
    SetString(Result, Buffer, Len);
  finally
    LocalFree(HLOCAL(Buffer));
  end;
end;

function Https_Post(const ServerName,Resource: String;const PostData : AnsiString;Var Response:AnsiString): Integer;
const
  BufferSize=1024*64;
var
  hInet    : HINTERNET;
  hConnect : HINTERNET;
  hRequest : HINTERNET;
  ErrorCode : Integer;
  lpdwBufferLength: DWORD;
  lpdwReserved    : DWORD;
  dwBytesRead     : DWORD;
  lpdwNumberOfBytesAvailable: DWORD;
begin
  Result  :=0;
  Response:='';
  hInet := InternetOpen(PChar(sUserAgent), INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);

  if hInet=nil then
  begin
    ErrorCode:=GetLastError;
    raise Exception.Create(Format('InternetOpen Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
  end;

  try
    hConnect := InternetConnect(hInet, PChar(ServerName), INTERNET_DEFAULT_HTTPS_PORT, nil, nil, INTERNET_SERVICE_HTTP, 0, 0);
    if hConnect=nil then
    begin
      ErrorCode:=GetLastError;
      raise Exception.Create(Format('InternetConnect Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
    end;

    try
      hRequest := HttpOpenRequest(hConnect, 'POST', PChar(Resource), HTTP_VERSION, '', nil, INTERNET_FLAG_SECURE, 0);
      if hRequest=nil then
      begin
        ErrorCode:=GetLastError;
        raise Exception.Create(Format('HttpOpenRequest Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
      end;

      try
        //send the post request
        if not HTTPSendRequest(hRequest, nil, 0, @PostData[1], Length(PostData)) then
        begin
          ErrorCode:=GetLastError;
          raise Exception.Create(Format('HttpSendRequest Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
        end;

          lpdwBufferLength:=SizeOf(Result);
          lpdwReserved    :=0;
          //get the response code
          if not HttpQueryInfo(hRequest, HTTP_QUERY_STATUS_CODE or HTTP_QUERY_FLAG_NUMBER, @Result, lpdwBufferLength, lpdwReserved) then
          begin
            ErrorCode:=GetLastError;
            raise Exception.Create(Format('HttpQueryInfo Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
          end;

         //if the response code =200 then get the body
         if Result=200 then
          if InternetQueryDataAvailable(hRequest, lpdwNumberOfBytesAvailable, 0, 0) then
          begin
            SetLength(Response,lpdwNumberOfBytesAvailable);
            InternetReadFile(hRequest, @Response[1], lpdwNumberOfBytesAvailable, dwBytesRead);
          end
          else
          begin
            ErrorCode:=GetLastError;
            raise Exception.Create(Format('InternetQueryDataAvailable Error %d Description %s',[ErrorCode,GetWinInetError(ErrorCode)]));
          end;

      finally
        InternetCloseHandle(hRequest);
      end;
    finally
      InternetCloseHandle(hConnect);
    end;
  finally
    InternetCloseHandle(hInet);
  end;
end;

function URLEncode(const Url: string): string;
var
  i: Integer;
begin
  Result := '';
  for i := 1 to Length(Url) do
  begin
    case Url[i] of
      'A'..'Z', 'a'..'z', '0'..'9', '-', '_', '.':
        Result := Result + Url[i];
    else
        Result := Result + '%' + IntToHex(Ord(Url[i]), 2);
    end;
  end;
end;

Procedure TestPost(const UrlList : Array of AnsiString);
var
 Response     : AnsiString;
 ResponseCode : Integer;
 Data         : AnsiString;
 i            : integer;
 LstUrl       : TStringList;
begin
   //create the body request with the url to lookup
   Data:=AnsiString(IntToStr(Length(UrlList)))+#10;
   for i:= low(UrlList) to high(UrlList) do
     Data:=Data+UrlList[i]+#10;

   //make the post request
   ResponseCode:=Https_Post(sServer,Format(sPostSafeBrowsing,[sApiKey]), Data, Response);

   //process the response
   case ResponseCode of
     200:
          begin
             LstUrl:=TStringList.Create;
             try
               LstUrl.Text:=string(Response);
                for i:=0 to  LstUrl.Count-1  do
                 Writeln(Format('The queried URL (%s) is %s',[UrlList[i],LstUrl[i]]));

             finally
               LstUrl.Free;
             end;
          end;
     204: Writeln('NONE of the queried URLs matched the phishing or malware lists, no response body returned');
     400: Writeln('Bad Request — The HTTP request was not correctly formed.');
     401: Writeln('Not Authorized — The apikey is not authorized');
     503: Writeln('Service Unavailable — The server cannot handle the request.');
   else
         Writeln(Format('Unknow response Code (%d)',[ResponseCode]));
   end;
end;

begin
  try
     //check these three urls at once
     TestPost(['orgsite.info','http://www.google.com','http://malware.testing.google.test/testing/malware/']);
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
  Readln;
end.

finally the result will be

The queried URL (orgsite.info) is malware
The queried URL (http://www.google.com) is ok
The queried URL (http://malware.testing.google.test/testing/malware/) is malware

Author: Rodrigo

Just another Delphi guy.

6 thoughts on “Using the Google Safe Browsing API from Delphi”

Tom Conlon
July 12, 2011 at 5:25 am

nice one, thanks for this Rodrigo.
Tom

A. Bouchez
July 12, 2011 at 9:01 am

Nice article..
But why do you still use WinINet?
This API is dead slow, and won’t work in a Windows Service. If you look for performance, take a look at WinHTTP, which is much faster than WinINet. More than 10 times faster, at least for multiple connections. Only missing feature is the dialog boxes for remote dial-up access. See http://msdn.microsoft.com/en-us/library/aa384273%28v=vs.85%29.aspx
I’ve implemented both WinInet and WinHTTP client access in our Open Source ORM framework. You may take a look at http://blog.synopse.info/post/2011/07/04/WinINet-vs-WinHTTP to find out more info about WinHTTP.
As far as I know, the latest version of IE uses WinHTTP instead of WinINet. So we may consider going in the same direction.
And both API are very close. So it won’t be difficult to convert to the “new style”. ;)

- Rodrigo
  July 12, 2011 at 5:21 pm
  
  Thanks for your comments, about your question
  
  But why do you still use WinINet?
  
  I still using WinInet because is the recomended for desktop applications , unless you plan to run within a service . About the perfomance you can’t compare both , because these interfaces was designed for different scenarios WinInet for client applications and WinHTTP for server side (but sure you can use WinHTTP in desktop application too) ,you must read this article WinHTTP vs. WinINet which says
  
  With a few exceptions, WinINet is a superset of WinHTTP. When selecting between the two, you should use WinINet
  
  , Also I use WinInet in this article to show how make a GET and POST request using WinInet :).
  
  - A. Bouchez
    July 13, 2011 at 2:20 am
    
    As I stated, latest version of IE uses WinHTTP instead of WinINet. ;)
    Microsoft also states in this article some benefits of WinHTTP… For the purpose of your article (using a remote API), using WinHTTP do make perfectly sense. A server is very likely to use such API.
    
Pingback: 使用Google Safe Browsing API - BccSafe's Blog
Steve
December 12, 2018 at 12:23 pm

Thank you so much for this. I’m writing a WinInet application and your sample code led me to find what was wrong with me code and why it would only connect with HTTP not HTTPS. Very grateful to you.