Organizational Research By

Surprising Reserch Topic

Question:Is there a good open source search engine including indexing bot which can be used to make up special catalogue by feeding the bot with certain properties?

Our application (C#/.NET) needs a lot of queries to search. Google's 50,000 policy per day is not enough. We need something that would crawl Internet websites by specific rules we set (for ex. country domains) and gather URLs, Texts, keywords, name of websites and create our own internal catalogue so we wouldn't be limited to any massive external search engine like Google or Yahoo.

Is there any free open source solution we could use to install it on our server?

No point in re-inventing the wheel.


asked Sep 13, 2013 in Crawl by rajesh
edited Sep 12, 2013
0 votes

Related Hot Questions

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
Anti-spam verification:
To avoid this verification in future, please log in or register.