FastPrepCrawl Reachable Pages on the Starting Host

Crawl Reachable Pages on the Starting Host

Google logoGoogle● MediumFULLTIMEONSITE INTERVIEW
Learn

Problem statement

Implement a web crawler over a supplied finite page graph. Start at startUrl and follow outgoing links, visiting each URL at most once. Crawl only pages whose hostname exactly matches the hostname of startUrl.

pageUrls[i] identifies a page and outgoing[i] lists the URLs linked from that page. Every linked URL appears in pageUrls. URLs use lowercase http:// or https:// followed by a hostname and optional path. The hostname is the text between :// and the next /, or the end of the URL. Compare hostnames exactly; scheme and path do not affect the host match.

Return all reachable same-host URLs, including startUrl, once each in lexicographic order. Links to another host are never followed. The supplied graph stands in for an HTML parser; this exercise makes no network requests.

Function

crawlSameHost(startUrl: String, pageUrls: String[], outgoing: String[][]) → String[]

Examples

Example 1

startUrl = "https://alpha.test/home"pageUrls = ["https://alpha.test/home","https://alpha.test/a","https://alpha.test/b","https://other.test/away"]outgoing = [["https://alpha.test/a","https://other.test/away"],["https://alpha.test/b","https://alpha.test/home"],["https://alpha.test/a"],["https://alpha.test/b"]]return = ["https://alpha.test/a","https://alpha.test/b","https://alpha.test/home"]

Only the three reachable pages on alpha.test are returned. The link to another host is ignored.

Example 2

startUrl = "http://site.test/root"pageUrls = ["http://site.test/root","http://site.test/a","http://site.test/hidden"]outgoing = [["http://site.test/a","http://site.test/a"],["http://site.test/root"],[]]return = ["http://site.test/a","http://site.test/root"]

The duplicate link and cycle are visited once; the disconnected page is not reached.

Example 3

startUrl = "https://a.test/"pageUrls = ["https://a.test/","http://a.test/path","https://b.test/"]outgoing = [["http://a.test/path","https://b.test/"],[],[]]return = ["http://a.test/path","https://a.test/"]

Matching hostnames are allowed even when the URL schemes differ. The other host is excluded.

Constraints

  • 1 <= pageUrls.length <= 1000 and outgoing.length == pageUrls.length.
  • startUrl appears exactly once in pageUrls; all page URLs are distinct.
  • Every link in outgoing appears in pageUrls; there are at most 5000 links in total, including duplicates.
  • Each URL is 1 through 100 characters and follows the stated lowercase scheme/hostname format. Hostnames are nonempty.

More Google problems

See Google hiring insights
public String[] crawlSameHost(String startUrl, String[] pageUrls, String[][] outgoing) {
    // Write your code here.
}
startUrl"https://alpha.test/home"
pageUrls["https://alpha.test/home","https://alpha.test/a","https://alpha.test/b","https://other.test/away"]
outgoing[["https://alpha.test/a","https://other.test/away"],["https://alpha.test/b","https://alpha.test/home"],["https://alpha.test/a"],["https://alpha.test/b"]]
expected["https://alpha.test/a", "https://alpha.test/b", "https://alpha.test/home"]
Checking account…