How can I search through Stack Overflow questions from a script?
给定一系列关键字，例如"python最佳实践"，我想获得包含该关键字的前10个堆栈溢出问题，这些问题按相关性(？)排序。，比如说从一个python脚本。我的目标是最终得到一个元组列表(标题、URL)。
我怎样才能做到这一点？你会考虑查询谷歌吗？(从python怎么做？)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | >>> from urllib import urlencode >>> params = urlencode({'q': 'python best practices', 'sort': 'relevance'}) >>> params 'q=python+best+practices&sort=relevance' >>> from urllib2 import urlopen >>> html = urlopen("http://stackoverflow.com/search?%s" % params).read() >>> import re >>> links = re.findall(r'([^<]*)', html) >>> links [('/questions/5119/what-are-the-best-rss-feeds-for-programmersdevelopers#5150', 'What are the best RSS feeds for programmers/developers?'), ('/questions/3088/best-ways-to-teach-a-beginner-to-program#13185', 'Best ways to teach a beginner to program?'), ('/questions/13678/textual-versus-graphical-programming-languages#13886', 'Textual versus Graphical Programming Languages'), ('/questions/58968/what-defines-pythonian-or-pythonic#59877', 'What defines “pythonian” or “pythonic”?'), ('/questions/592/cxoracle-how-do-i-access-oracle-from-python#62392', 'cx_Oracle - How do I access Oracle from Python? '), ('/questions/7170/recommendation-for-straight-forward-python-frameworks#83608', 'Recommendation for straight-forward python frameworks'), ('/questions/100732/why-is-if-not-someobj-better-than-if-someobj-none-in-python#100903', 'Why is if not someobj: better than if someobj == None: in Python?'), ('/questions/132734/presentations-on-switching-from-perl-to-python#134006', 'Presentations on switching from Perl to Python'), ('/questions/136977/after-c-python-or-java#138442', 'After C++ - Python or Java?')] >>> from urlparse import urljoin >>> links = [(urljoin('http://stackoverflow.com/', url), title) for url,title in links] >>> links [('http://stackoverflow.com/questions/5119/what-are-the-best-rss-feeds-for-programmersdevelopers#5150', 'What are the best RSS feeds for programmers/developers?'), ('http://stackoverflow.com/questions/3088/best-ways-to-teach-a-beginner-to-program#13185', 'Best ways to teach a beginner to program?'), ('http://stackoverflow.com/questions/13678/textual-versus-graphical-programming-languages#13886', 'Textual versus Graphical Programming Languages'), ('http://stackoverflow.com/questions/58968/what-defines-pythonian-or-pythonic#59877', 'What defines “pythonian” or “pythonic”?'), ('http://stackoverflow.com/questions/592/cxoracle-how-do-i-access-oracle-from-python#62392', 'cx_Oracle - How do I access Oracle from Python? '), ('http://stackoverflow.com/questions/7170/recommendation-for-straight-forward-python-frameworks#83608', 'Recommendation for straight-forward python frameworks'), ('http://stackoverflow.com/questions/100732/why-is-if-not-someobj-better-than-if-someobj-none-in-python#100903', 'Why is if not someobj: better than if someobj == None: in Python?'), ('http://stackoverflow.com/questions/132734/presentations-on-switching-from-perl-to-python#134006', 'Presentations on switching from Perl to Python'), ('http://stackoverflow.com/questions/136977/after-c-python-or-java#138442', 'After C++ - Python or Java?')] |
将其转换为函数应该很简单。
编辑：见鬼，我会的…
1 2 3 4 5 6 7 8 | def get_stackoverflow(query): import urllib, urllib2, re, urlparse params = urllib.urlencode({'q': query, 'sort': 'relevance'}) html = urllib2.urlopen("http://stackoverflow.com/search?%s" % params).read() links = re.findall(r'([^<]*)', html) links = [(urlparse.urljoin('http://stackoverflow.com/', url), title) for url,title in links] return links |
由于stackoverflow已经有了这个特性，您只需要获取搜索结果页面的内容并获取所需的信息。以下是按相关性搜索的URL:
https://stackoverflow.com/search?q=python+best+practices&sort=relevance
如果查看源代码，您将看到每个问题所需的信息都位于这样的行中：
1 | What are the best RSS feeds for programmers/developers? |
所以您应该能够通过对该表单的字符串执行regex搜索来获得前十个。
建议向SO添加一个REST API。http://stackoverflow.uservoice.com/
您可以从一个有效的HTTP请求中截取返回的HTML。但这将导致坏业力，以及失去享受良好睡眠的能力。
我只需要使用pycurl将搜索词连接到查询URI。