Auto Search Sites And Send Alerts

[alocke79]

As a consultant, I sometimes have requests for interesting apps. I have seen a couple of apps that do this, mainly for mobile phones, and all specific to Craigslist.

The goal of my small business client, that is mostly a reseller, is to have an application that searches open commerce forums such as Craigslist, Backpage.com, eBay, etc. Once a keyword is found, such as "ipad", the application will send a notification with a link. The notification can be email based, txt message, or even a web interface that refreshes frequently and uses some basic popup.

Not being a coder (yet), I'm looking for third party apps that can do this. Or, since this is not a rush job, advice on a language and reference material I can use to make this a learning project for myself.

Thanks in advance,

Alex

[digip]

Depending on how this would be done, I can think of a few ways to start it off. One would be a cron job or scheduled job that starts a page, php or curl script that scrapes sites you tell it to check. Now, something like craigslist, I guess you would need to spider its results, say from a pre made search url, then spider links on the page, index them for record purposes so not to check them every visit and only look for new unique posts, then scrape those pages, search your terms, then email you the page its found on. Doesn't sound too difficult, but I'm also not a programmer, but I imagine wouldn't take someone too long to put it all together.

You could even just have it as a static php page that you visit whenever you want to check for results ahead of schedule to check your search phrases, and it would run whenever you visit the url manually.

[moonlit]

Some combination of RSS feeds (like this or this or this), Yahoo Pipes and an RSS to email service (or app) like this one would probably do the trick, no coding req'd.

Edited July 24, 2010 by moonlit

[Sparda]

http://www.google.com/alerts would be a good starting point, though you would have to create a alert for each site as google only allows you to search a single domain at a time (i believe).