Skip to main content

Use Desktop Crawler App for AMP

  • Updated

Before you can use Desktop Crawler App, you must download and install it on your computer. After that, log in to the environment you want to scan, log in to Desktop Crawler App with your AMP URL and API key (token), select an organization and asset within it that you want to scan, and then run a scan.

On this page:

Add specific URLs to allowlist

DCA will access some resources on the public internet for normal function. If your network security blocks access to these resources, you will need to work with your security team to add them to an allowlist. 

Add the following URLs to the allowlist: 

  • api2.amplitude.com: Use  to track user interaction via Amplitude.  
  • amp.levelaccess.net or company-specific AMP instances: DCA pulls best practices from AMP to add details to the results data. Data is pulled from whatever instance of AMP DCA users are connecting to. For many users that is amp.levelaccess.net but your company may have its own AMP instances. 
  • Instance of AMP that DCA is configured to push results to: You must be able to reach your instances of AMP. DCA can’t push results to an instance it can’t reach. The initial setup where you choose an organization  and  assets won’t happen if DCA can’t reach the configured instance.

Download and install Desktop Crawler App (DCA)

Depending on your web browser, downloading instructions might differ slightly. 

Download and install DCA for Windows using MS Edge

To download Desktop Crawler App for Windows using Microsoft Edge browser:

  1. Log into AMP.
  2. Go to Toolbox.
  3. Find Desktop Crawler App.
  4. Select Download installer for Windows. The progress bar opens, asking you to make sure that you trust DesktopCrawlerApp.msi file. 
  5. Select the three-dot action menu in the top corner of the progress bar. 
  6. Select Keep. A new message opens, prompting you to reconfirm your choice. 
  7. Select Keep anyway. The DesktopCrawlerApp.msi file is downloaded to your desktop.
  8. Run the file to install the application. 
  9. Follow the installer’s prompts to complete the installation. 

Download and install DCA for Windows using Google Chrome

To download and install Desktop Crawler App for Windows using Chrome browser:

  1. Log into AMP.
  2. Go to Toolbox.
  3. Find Desktop Crawler App.
  4. Select Download installer for Windows.  The installer is downloaded.
  5. Run the installer. A message opens. 
  6. Select More info. 
  7. Select Run anyway to install the application.
  8. Follow the installer’s prompts to complete the installation. 

You can now log in to the app with your AMP URL and API token.

Note: Some IT departments do not allow downloading of unsigned .msi files. In this situation, ask the IT administrator to add an exception to your machine.

Log in to Desktop Crawler App

To log in to Desktop Crawler App:

  1. Open Desktop Crawler App.
  2. Enter your AMP URL*. The  AMP URL value should be set to the URL of your AMP instance. In most situations, this will be https://amp.levelaccess.net/. If your organization uses a different URL to access AMP, enter that URL instead. 
  3. In the API Key* field, enter the corresponding API key (token). 
    Learn more about creating an AMP API key (token).  The API key (token) should be the one from your account in AMP. This authorizes Desktop Crawler App to push results to AMP. 
  4. Select Log in.

You're ready to scan with Desktop Crawler App.

Scan with Desktop Crawler App

Desktop Crawler App can scan up to 250 pages with a crawl depth of 25.  

Run a scan using Desktop Crawler App

To scan with Desktop Crawler App:

  1. Log in to DCA.
  2. On the Assets screen, from the Organization dropdown, select an organization.
    • If you have access to more than one organization the first is chosen by default. 
    • The organization contains the asset you want to use to save your scan results.
    • Assets you can access in this organization are listed.  
  3. Find the asset you want to scan. You must have Editor rights on an asset for it to be available on this list. Viewer rights are not sufficient. 
  4. Select Create scan.
  5. Enter or select the values for these settings:
    • Report name.
    • Website URL.
    • Maximum number of pages.
    • Crawl depth.
    • Skip URL:
      • # endings
      • ? endings
    • Add webpages to exclude
  6. Select Authenticate to validate scan setup form entries if you are scanning a protected site, behind a security wall. Skip this step if you are scanning a public site. When you select the Authenticate button, DCA launches a Chrome browser window and loads the URL specified in DCA as the starting URL. It asks you to log in if the URL is protected. If needed, complete the login.
  7. Select Run scan.

What happens when you run a scan? 

The application starts crawling and scanning the pages using the settings you specified. It discovers, scans, and submits results in a sequential manner. Results are pushed to AMP after each page is scanned.
Closing the application will interrupt the scanning process, and only partial results will be available in AMP. Scanning will stop when:  

  • All discovered possible URLs are scanned, or 
  • The maximum number of pages is reached.

Once the scan is complete,  the results report will be available in your AMP instance. You can view the report in AMP or create a new scan. Note that the score is populated asynchronously and may not show immediately after the scan is complete. 

To open the results from this DCA scan in your browser:

  • Select the View report in AMP button. 

Cancel a scan 

You can cancel a scan for any reason.

To cancel a scan: 

  1. Select the Cancel Scan button.
  2. Confirm that you want to cancel the scan. Once you do, the application will  stop scanning.

iframes and Desktop Crawler App

Many websites use iframes to embed another page inside a page, such as a social media widget. Desktop Crawler App always includes findings from iframes in the results. 

Note the following guidelines:

  • Best practice is to always have iFrame content on. 
  • By default, DCA scans iframe content and merges the results into the main page report. Issues from iframes are combined with the parent page results. 
  • In AMP reports, when an issue is flagged inside an iframe, the issue is also given special notation in the locator (||).  That is, element paths from iframes use special notation to show where they came from.  

Understanding configuration settings for crawling

Before you can run a scan, you need to specify values for the following settings: 

Setting Value
Report Name

Name of the report that will be created in AMP. Includes an appended time stamp. 

Example:

  • Report Name: Edge Not Scan
  • Rendered as: Edge Not Scan (2025-06-06T17:51:51.994Z).

The Z stands for Zulu time and not your particular time zone.
Website URL

The page on which the spider will start. Start location. 

Example: www.mycompany.com/foo

Note: If a link takes you to a page that normally would be in scope for being scanned but which is a redirect to a page that is not in scope, that page will not be scanned. For example, if your starting URL was www.mycompany.com/foo and it has a link to www.mycompany.com/foo/moreinfo but the moreinfo page redirects to www.mycompany.com/bar, that final destination page will not be scanned. That is,  /foo/moreinfo will not be scanned because there is no page there. It is a redirect to somewhere else. If that somewhere else is /bar, that also will not be scanned because it doesn’t start with /foo.  
Maximum number of pages

The number of pages to scan or test. The maximum value is 250 pages.

Example: 50
Crawl depth

The depth of a webpage in the website’s hierarchical structure. Indicates how many sub-levels you'd like to test. Depth aligns with the number of backslashes in a URL. The minimum is 1. The maximum is 25. 

Example of Crawl depth value: 5

Note: If we set Crawl depth to 5 and we scan a page that took four links to reach, we are already five levels deep. After testing that page, the scanner will not look for any more links on that page to honour the limit on Crawl depth. Instead, the crawler moves to other pages in its queue.
Skip URL # 

Optional setting. Both Skip options default to off, which is the same as the default for AMP spiders.

  • # ending: Some URLs will have a "fragment identifier” starting with a # to jump directly to certain sections of a page such as www.foo.com#body. These fragment identifiers often indicate different states of the page. The page may present different content depending on the presence of certain fragment identifiers in the URL. Select this checkbox if your site does not use fragment identifiers to trigger different states and content that you want scanned separately, as unique pages
Skip URL ? 

Optional setting. Both Skip options default to off, which is the same as the default for AMP spiders.

  • ? endings: Some URLs will have one or more “URL query string parameters” starting with a ? and separated by an & to track various inputs needed to control a page state. These might be pagination, a catalog item ID, search terms or anything the site’s developers wanted to store in the URL. For example, you could the URL can look like this: www.foo.com?sort=newest&color=blue. These URL query string parameters have meaning only to the code running this site and may make substantial changes to the displayed content. Select this checkbox if your site does not use URL query string parameters to trigger different states and content that you want scanned separately, as unique pages.
Add webpages to exclude

Ignores URLs we want to skip when running a scan. Select the checkbox, and add comma separated URLs you want DCA to ignore when running a scan.  For example, if you want to ignore the Logout link/page, add its URL to the list of pages to ignore.  You can also specify these URLs as regular expressions.  

Troubleshoot Desktop Crawler App for AMP

Why did the scan fail?  

Scans can fail if the website is not reachable, it takes too long to respond, or if the computer loses network connectivity. 

My scan did not reach crawl depth I specified?

Desktop Crawler App can scan up to 250 pages with a crawl depth of 25. You must ensure that Crawl depth aligns with the number of backslashes in a URL. The minimum is 1. The maximum is 25. 

Why do my findings and scores differ between AMP and DCA?

DCA and AMP page selection algorithms might choose different sets of pages to test. For example, if a site has 100 pages but only 10 are scanned, the chosen set of pages might vary between AMP and DCA. Consequently, the findings and scores derived from those findings might differ between AMP and DCA. However, they will be identical if you scan all the pages, because  page selection no longer influences the outcome. 

Related articles

Comments

0 comments

Article is closed for comments.