Data acquisition

详细信息

We are a software company. For one of our projects we need to downloadinformation from a website containing articles about medical topics.The website contains cca. 10000 HTML pages of paged listing of articlesin Czech language. The list contains titles of articles, each title havinga link to the detail HTML page with the article text.We need someone to produce wget and other scripts and download the titles ofall articles, parse the links from those titles, download the detailed pagesof the articles and distill the text that is shown in the page. The titles as well as the detail pages mostly have the same structure sothis allows for an automated work. But it is not so in 100% cases, there maybe several types of structure so it may require some attention as to howto distill the correct information.The result of this work will be a set of static HTML files. You can view thisstructure underhttps://fomenot.com/z/dwld24/main.htmlI.e. the result will contain the contents of the article separated intoparagraphs of normal text and captions (nothing else, no images or othertexts). We only want the main text of the article that is visible on the screenfor the user. No other text or html content.Another result will be the raw HTML output for each of the detail pagesFor accepting the output, we will do our check of the result. If we find errors,we will give examples of these errors and we will expect the vendor to fixall such errors in the result, not just those examples. If there are only a fewerrors we may not be able to find them and it is ok. But if we find any we willrequire correcting them.We expect that the raw HTML files will be 100% error free (for these we will notgive examples, we just would demand fixing them). For the text-based resultswe will give examples before demanding to fix them.
An example of such a source page you can find here: https://www.idnes.cz/onadnes/zdravi/2You can see a list of articles, each having a link leading to the detailand then a paging control that can load more articles from the next page.This is NOT the page we need to download but similar. Putting here the exampleonly that you understand what is the task.
Let us know if you could do it and for what price. We will provide the real linksto the selected candidate.