It also guarantees that there is a subset of domains thats basically feasible to crawl in affordable time body. I then additional in some logic to rank matches in domains/urls and titles better than written content, penalize scaled-down matching files and reward lengthier kinds (to offset the bias thats in BM25). I did not want to overthink this so I executed BM25 position for the key ranking calculation.