Ucnt Posted February 24, 2013 Share Posted February 24, 2013 (edited) Desired end state: Scan and scrape a range of IP addresses, looking for a router model within an an HTML page, and output the IP address, if found. Note: Multiple versions of the router are open and the xpaths to the hardware versions are different... Background: I've been goofing around with nmap, scanning a few subnets around my IP address (Comcast) and have noticed that 5-10 Belkin F5D model routers per class B subnet are open on port 80 and about half of them have no admin password established. I want to do a more extensive scan but want to limit the IPs returned to the F5D routers, not every device open on port 80 so I don't have to go to each page and manually identify them. From what I understand python doesn't work well with nmap and I am not sure a good way to approach this and I was struggling to get something like BeautifulSoup to successfully scrape a config page. Any ideas? Edited February 24, 2013 by whitehatvet Quote Link to comment Share on other sites More sharing options...
airman_dopey Posted February 24, 2013 Share Posted February 24, 2013 One hopes that this would be your ip address (although I would not post mine on a "hacking" website) because it would be really shitty if you were advertising a stranger's ip with hardware listed. Quote Link to comment Share on other sites More sharing options...
Ucnt Posted February 24, 2013 Author Share Posted February 24, 2013 Not mine but touche I guess? or at least I understand what you are saying....though it's trivial to find an open router's config page that is password protected. It's not inviting the hacking of that router, it's using it as a reference in order to parse HTML data from. If I posted one of the open IP addresses, I'd be more apt to agree with you... That being said maybe a 'better' way to show this is to show you an xpath to one of the routers, written two different ways. id('cont_table')/x:tbody/x:tr/x:td[2]/x:blockquote/x:form/x:div[1]/x:table[1]/x:tbody/x:tr[2]/x:td[2]#<td> /tbody/tr/td[2]/blockquote/form/div[1]/table[1]/tbody/tr[2]/td[2]/text() Quote Link to comment Share on other sites More sharing options...
Ucnt Posted February 24, 2013 Author Share Posted February 24, 2013 Oh, and I have tried BeautifulSoup using various methods (searching for the title, links, etc) but it is not reading the Belkin router's config page properly. Quote Link to comment Share on other sites More sharing options...
pasteeywhitecoder Posted February 24, 2013 Share Posted February 24, 2013 If you can't find something that works, make it yourself. I know, just by what you listed, I could write a program to do that in Java. Quote Link to comment Share on other sites More sharing options...
Ucnt Posted February 24, 2013 Author Share Posted February 24, 2013 That's what I'm trying to do...I'm not looking for someone to hand me a turn key solution... Though I'm new to coding, I have already written code, though admittedly easy and probably inefficient, that identifies IPs with port 80 open and exports the results to a text file and script (for automatically opening them in firefox) so I can browse through them. That is obviously is time intensive so I tried to use BeatifulSoup and similar methods to scrape the Belkin page, as I said above, but none work. There is no title to the page (just the IP), links are not being detected, and I have been unable to specify the hardware field to specifically parse that data. I understand that you and others can write code to do it; I'm just looking for advice on how you would go about it since the methods I have tried failed... Quote Link to comment Share on other sites More sharing options...
pasteeywhitecoder Posted February 24, 2013 Share Posted February 24, 2013 Well, here's what I would do. I would execute nmap through my Java application (look up Java Process object and reading from the InputStream). From the output of nmap, I would determine if an IP has port 80 open. If so, I would make an HTTP Request for that IP and store the result into a string. Then, I would parse the string for the information I need. Finally, display it on the interface. Quote Link to comment Share on other sites More sharing options...
Ucnt Posted February 24, 2013 Author Share Posted February 24, 2013 Thanks for the feedback. I'm only in my 2nd semester of Java and didn't know you could use it to execute nmap or do an HTTP request like that. I appreciate it and I'll start playing around with it tonight. Sounds like it should be fairly easy... Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.