Crawl Reachable Pages on the Starting Host
Problem statement
Implement a web crawler over a supplied finite page graph. Start at startUrl and follow outgoing links, visiting each URL at most once. Crawl only pages whose hostname exactly matches the hostname of startUrl.
pageUrls[i] identifies a page and outgoing[i] lists the URLs linked from that page. Every linked URL appears in pageUrls. URLs use lowercase http:// or https:// followed by a hostname and optional path. The hostname is the text between :// and the next /, or the end of the URL. Compare hostnames exactly; scheme and path do not affect the host match.
Return all reachable same-host URLs, including startUrl, once each in lexicographic order. Links to another host are never followed. The supplied graph stands in for an HTML parser; this exercise makes no network requests.
Function
crawlSameHost(startUrl: String, pageUrls: String[], outgoing: String[][]) → String[]Examples
Example 1
startUrl = "https://alpha.test/home"pageUrls = ["https://alpha.test/home","https://alpha.test/a","https://alpha.test/b","https://other.test/away"]outgoing = [["https://alpha.test/a","https://other.test/away"],["https://alpha.test/b","https://alpha.test/home"],["https://alpha.test/a"],["https://alpha.test/b"]]return = ["https://alpha.test/a","https://alpha.test/b","https://alpha.test/home"]Only the three reachable pages on alpha.test are returned. The link to another host is ignored.
Example 2
startUrl = "http://site.test/root"pageUrls = ["http://site.test/root","http://site.test/a","http://site.test/hidden"]outgoing = [["http://site.test/a","http://site.test/a"],["http://site.test/root"],[]]return = ["http://site.test/a","http://site.test/root"]The duplicate link and cycle are visited once; the disconnected page is not reached.
Example 3
startUrl = "https://a.test/"pageUrls = ["https://a.test/","http://a.test/path","https://b.test/"]outgoing = [["http://a.test/path","https://b.test/"],[],[]]return = ["http://a.test/path","https://a.test/"]Matching hostnames are allowed even when the URL schemes differ. The other host is excluded.
Constraints
1 <= pageUrls.length <= 1000andoutgoing.length == pageUrls.length.startUrlappears exactly once inpageUrls; all page URLs are distinct.- Every link in
outgoingappears inpageUrls; there are at most 5000 links in total, including duplicates. - Each URL is 1 through 100 characters and follows the stated lowercase scheme/hostname format. Hostnames are nonempty.