Here is a challenge:
We are developing an application that involves looking up specific data (values) from an active web page and bring those values in to our external application and do some data lookups with it independently.
Say our application is a system tray app or a windows forms C# app that will need to look up an active web page on (any) browser and look for two values – last name and date of birth. we will need to do possibly html view source of the web page to look up those
two values (for last name and date of birth) and then grab the value to bring it into our C# app to do further computation.
Note that the web page is not static and it is part of an application that our user will be logged into. but we can assume that lastname and dateofbirth will be present as part of the screen/page. so traditional web scraping might not work.
Any help on how we can accomplish this would be greatly appreciated.
If you can get active page links, or if you have those page links already, then you can download content using WebClient, and use string manipulation or regular expression to get value.
System.Net.WebClient wc = new System.Net.WebClient(); string viewsourcecontent = wc.DownloadString("URL");
Thanks So much for the response. This is a good suggestion, the only difficulty is, as you predicted, page links. Page links on this application that we are trying to scrap data from is not static. The links are dynamic – that is, we may not have the entire
URL and feed it to the wc.DownloadString("URL")
Given that the URL may be dynamic, are there any other ways to deal with this?