How to use java.net.URLConnection to fire and handle HTTP requests
这里经常会问到使用
该教程基本上只显示了如何触发GET请求和读取响应。它不解释在任何地方如何使用它来执行POST请求、设置请求头、读取响应头、处理cookie、提交HTML表单、上载文件等。
那么,如何使用
首先是免责声明:发布的代码片段都是基本的例子。你需要处理一些琐碎的
我们首先至少需要知道URL和字符集。参数是可选的,取决于功能需求。好的。
1 2 3 4 5 6 7 8 9 | String url ="http://example.com"; String charset ="UTF-8"; // Or in Java 7 and later, use the constant: java.nio.charset.StandardCharsets.UTF_8.name() String param1 ="value1"; String param2 ="value2"; // ... String query = String.format("param1=%s¶m2=%s", URLEncoder.encode(param1, charset), URLEncoder.encode(param2, charset)); |
查询参数必须为
这是一项微不足道的任务。这是默认的请求方法。好的。
1 2 3 4 | URLConnection connection = new URL(url +"?" + query).openConnection(); connection.setRequestProperty("Accept-Charset", charset); InputStream response = connection.getInputStream(); // ... |
任何查询字符串都应该使用
1 2 |
不管怎样,如果另一方是
出于测试目的,您可以将响应正文打印到stdout,如下所示:好的。
1 2 3 4 |
使用查询参数触发HTTP POST请求
将
1 2 3 4 5 6 7 8 9 10 11 | URLConnection connection = new URL(url).openConnection(); connection.setDoOutput(true); // Triggers POST. connection.setRequestProperty("Accept-Charset", charset); connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded;charset=" + charset); try (OutputStream output = connection.getOutputStream()) { output.write(query.getBytes(charset)); } InputStream response = connection.getInputStream(); // ... |
注意:当您想以编程方式提交HTML表单时,不要忘记将任何
您还可以将获得的
1 2 3 | HttpURLConnection httpConnection = (HttpURLConnection) new URL(url).openConnection(); httpConnection.setRequestMethod("POST"); // ... |
不管怎样,如果另一方是
您可以使用
HTTP响应状态:好的。
你需要一个
1 | int status = httpConnection.getResponseCode(); |
HTTP响应标头:
好。
HTTP响应编码:
好。
当
好。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | String contentType = connection.getHeaderField("Content-Type"); String charset = null; for (String param : contentType.replace("","").split(";")) { if (param.startsWith("charset=")) { charset = param.split("=", 2)[1]; break; } } if (charset != null) { try (BufferedReader reader = new BufferedReader(new InputStreamReader(response, charset))) { for (String line; (line = reader.readLine()) != null;) { // ... System.out.println(line) ? } } } else { // It's likely binary content, use InputStream/OutputStream. } |
维持的会议
在服务器端session冰通常支持一个饼干。一些Web形式的需要,你的记录和/或是在tracked市的一次会议。你可以使用API的
好。
1 2 3 4 5 6 7 8 9 10 11 12 | // First set the default cookie manager. CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL)); // All the following subsequent URLConnections will use the same cookie manager. URLConnection connection = new URL(url).openConnection(); // ... connection = new URL(url).openConnection(); // ... connection = new URL(url).openConnection(); // ... |
注意,这是已知的两个并不总是恰当的工作在所有情况下。如果这对你fails最好的冰淇淋,然后两个表壳,手动上弦收集和看过的Cookie标头。你需要两个翻盖basically全
好。
1 2 3 4 5 6 7 8 9 10 11 | // Gather all cookies on the first request. URLConnection connection = new URL(url).openConnection(); List<String> cookies = connection.getHeaderFields().get("Set-Cookie"); // ... // Then use the same cookies on all subsequent requests. connection = new URL(url).openConnection(); for (String cookie : cookies) { connection.addRequestProperty("Cookie", cookie.split(";", 2)[0]); } // ... |
"有两个
好。 流模式
《
好。
1 | httpConnection.setFixedLengthStreamingMode(contentLength); |
但如果内容长度beforehand冰真的不知道,我可以让你使用chunked流模式的
好。
1 | httpConnection.setChunkedStreamingMode(1024); |
用户代理
它可以发生,一个unexpected归来的请求应答,当它与一个真正的好作品的Web浏览器。我们的服务器端的冰堵的要求为基础的
好。
1 | connection.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"); // Do as if you're using Chrome 41 on Windows 7. |
使用"用户代理字符串从一个新的浏览器。
好。 错误的行为
如果HTTP响应代码
好。
1 |
如果HTTP响应代码是- 1,后来走错了与连接的行动和反应。《
好。
1 |
上传文件
对于混合的post内容(二进制和字符数据),通常使用
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | String param ="value"; File textFile = new File("/path/to/file.txt"); File binaryFile = new File("/path/to/file.bin"); String boundary = Long.toHexString(System.currentTimeMillis()); // Just generate some unique random value. String CRLF =" "; // Line separator required by multipart/form-data. URLConnection connection = new URL(url).openConnection(); connection.setDoOutput(true); connection.setRequestProperty("Content-Type","multipart/form-data; boundary=" + boundary); try ( OutputStream output = connection.getOutputStream(); PrintWriter writer = new PrintWriter(new OutputStreamWriter(output, charset), true); ) { // Send normal param. writer.append("--" + boundary).append(CRLF); writer.append("Content-Disposition: form-data; name="param"").append(CRLF); writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF); writer.append(CRLF).append(param).append(CRLF).flush(); // Send text file. writer.append("--" + boundary).append(CRLF); writer.append("Content-Disposition: form-data; name="textFile"; filename="" + textFile.getName() +""").append(CRLF); writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF); // Text file itself must be saved in this charset! writer.append(CRLF).flush(); Files.copy(textFile.toPath(), output); output.flush(); // Important before continuing with writer! writer.append(CRLF).flush(); // CRLF is important! It indicates end of boundary. // Send binary file. writer.append("--" + boundary).append(CRLF); writer.append("Content-Disposition: form-data; name="binaryFile"; filename="" + binaryFile.getName() +""").append(CRLF); writer.append("Content-Type:" + URLConnection.guessContentTypeFromName(binaryFile.getName())).append(CRLF); writer.append("Content-Transfer-Encoding: binary").append(CRLF); writer.append(CRLF).flush(); Files.copy(binaryFile.toPath(), output); output.flush(); // Important before continuing with writer! writer.append(CRLF).flush(); // CRLF is important! It indicates end of boundary. // End of multipart/form-data. writer.append("--" + boundary +"--").append(CRLF).flush(); } |
如果另一方是
有时候你需要连接一个https URL,也许是因为你在写一个web scraper。在这种情况下,您可能会在某些HTTPS站点上遇到不保持其SSL证书最新的
在Web scraper类中,下面的一次性运行
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | static { TrustManager[] trustAllCertificates = new TrustManager[] { new X509TrustManager() { @Override public X509Certificate[] getAcceptedIssuers() { return null; // Not relevant. } @Override public void checkClientTrusted(X509Certificate[] certs, String authType) { // Do nothing. Just allow them all. } @Override public void checkServerTrusted(X509Certificate[] certs, String authType) { // Do nothing. Just allow them all. } } }; HostnameVerifier trustAllHostnames = new HostnameVerifier() { @Override public boolean verify(String hostname, SSLSession session) { return true; // Just allow them all. } }; try { System.setProperty("jsse.enableSNIExtension","false"); SSLContext sc = SSLContext.getInstance("SSL"); sc.init(null, trustAllCertificates, new SecureRandom()); HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory()); HttpsURLConnection.setDefaultHostnameVerifier(trustAllHostnames); } catch (GeneralSecurityException e) { throw new ExceptionInInitializerError(e); } } |
最后的话
apache httpcomponents httpclient在这一切中更加方便:)好的。
- httpclient教程
- httpclient示例
分析和提取HTML
如果您只需要解析和提取HTML中的数据,那么最好使用像jsoup这样的HTML解析器。好的。
- 在Java中领先的HTML解析器的优点/缺点
- 如何在Java中扫描和提取网页
好啊。
在使用HTTP时,引用
然后,您可以不依赖于
它还提供有用的HTTP常量,因此您可以执行以下操作:
1 2 3 | int responseCode = httpURLConnection.getResponseCode(); if (responseCode == HttpURLConnection.HTTP_OK) { |
受此和其他问题的启发,我创建了一个最小的开源基本HTTP客户端,它体现了这里发现的大多数技术。
谷歌HTTP Java客户端也是一个伟大的开源资源。
有两个选项可以用于HTTP URL命中:get/post
获取请求:
1 2 3 4 5 6 7 8 9 | HttpURLConnection.setFollowRedirects(true); // defaults to true String url ="https://name_of_the_url"; URL request_url = new URL(url); HttpURLConnection http_conn = (HttpURLConnection)request_url.openConnection(); http_conn.setConnectTimeout(100000); http_conn.setReadTimeout(100000); http_conn.setInstanceFollowRedirects(true); System.out.println(String.valueOf(http_conn.getResponseCode())); |
职位要求:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | HttpURLConnection.setFollowRedirects(true); // defaults to true String url ="https://name_of_the_url" URL request_url = new URL(url); HttpURLConnection http_conn = (HttpURLConnection)request_url.openConnection(); http_conn.setConnectTimeout(100000); http_conn.setReadTimeout(100000); http_conn.setInstanceFollowRedirects(true); http_conn.setDoOutput(true); PrintWriter out = new PrintWriter(http_conn.getOutputStream()); if (urlparameter != null) { out.println(urlparameter); } out.close(); out = null; System.out.println(String.valueOf(http_conn.getResponseCode())); |
我建议您看一下kevinsawicki/http请求的代码,它基本上是
示例:使用内容类型
1 2 3 4 5 |
更新
The new HTTP Client shipped with Java 9 but as part of an
Incubator module namedjdk.incubator.httpclient . Incubator modules are
a means of putting non-final APIs in the hands of developers while the
APIs progress towards either finalization or removal in a future
release.
在Java 9中,可以发送一个EDCOX1×8请求,例如:
1 2 3 4 5 6 | // GET HttpResponse response = HttpRequest .create(new URI("http://www.stackoverflow.com")) .headers("Foo","foovalue","Bar","barvalue") .GET() .response(); |
然后检查返回的
1 2 | int statusCode = response.statusCode(); String responseBody = response.body(HttpResponse.asString()); |
由于这个新的HTTP客户端位于
1 2 3 | module com.foo.bar { requires jdk.incubator.httpclient; } |
我也受到了这种反应的启发。
我经常在需要做一些HTTP的项目上工作,我可能不想引入很多第三方依赖项(这会引入其他依赖项等等)。
我开始根据这段对话编写自己的实用程序(不是在任何地方完成的):
1 2 3 4 5 6 7 8 9 10 11 12 13 | package org.boon.utils; import java.io.IOException; import java.io.InputStream; import java.net.HttpURLConnection; import java.net.URL; import java.net.URLConnection; import java.util.Map; import static org.boon.utils.IO.read; public class HTTP { |
然后就有一堆或静态方法。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | public static String get( final String url) { Exceptions.tryIt(() -> { URLConnection connection; connection = doGet(url, null, null, null); return extractResponseString(connection); }); return null; } public static String getWithHeaders( final String url, final Map<String, ? extends Object> headers) { URLConnection connection; try { connection = doGet(url, headers, null, null); return extractResponseString(connection); } catch (Exception ex) { Exceptions.handle(ex); return null; } } public static String getWithContentType( final String url, final Map<String, ? extends Object> headers, String contentType) { URLConnection connection; try { connection = doGet(url, headers, contentType, null); return extractResponseString(connection); } catch (Exception ex) { Exceptions.handle(ex); return null; } } public static String getWithCharSet( final String url, final Map<String, ? extends Object> headers, String contentType, String charSet) { URLConnection connection; try { connection = doGet(url, headers, contentType, charSet); return extractResponseString(connection); } catch (Exception ex) { Exceptions.handle(ex); return null; } } |
然后张贴…
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 | public static String postBody( final String url, final String body) { URLConnection connection; try { connection = doPost(url, null,"text/plain", null, body); return extractResponseString(connection); } catch (Exception ex) { Exceptions.handle(ex); return null; } } public static String postBodyWithHeaders( final String url, final Map<String, ? extends Object> headers, final String body) { URLConnection connection; try { connection = doPost(url, headers,"text/plain", null, body); return extractResponseString(connection); } catch (Exception ex) { Exceptions.handle(ex); return null; } } public static String postBodyWithContentType( final String url, final Map<String, ? extends Object> headers, final String contentType, final String body) { URLConnection connection; try { connection = doPost(url, headers, contentType, null, body); return extractResponseString(connection); } catch (Exception ex) { Exceptions.handle(ex); return null; } } public static String postBodyWithCharset( final String url, final Map<String, ? extends Object> headers, final String contentType, final String charSet, final String body) { URLConnection connection; try { connection = doPost(url, headers, contentType, charSet, body); return extractResponseString(connection); } catch (Exception ex) { Exceptions.handle(ex); return null; } } private static URLConnection doPost(String url, Map<String, ? extends Object> headers, String contentType, String charset, String body ) throws IOException { URLConnection connection;/* Handle output. */ connection = new URL(url).openConnection(); connection.setDoOutput(true); manageContentTypeHeaders(contentType, charset, connection); manageHeaders(headers, connection); IO.write(connection.getOutputStream(), body, IO.CHARSET); return connection; } private static void manageHeaders(Map<String, ? extends Object> headers, URLConnection connection) { if (headers != null) { for (Map.Entry<String, ? extends Object> entry : headers.entrySet()) { connection.setRequestProperty(entry.getKey(), entry.getValue().toString()); } } } private static void manageContentTypeHeaders(String contentType, String charset, URLConnection connection) { connection.setRequestProperty("Accept-Charset", charset == null ? IO.CHARSET : charset); if (contentType!=null && !contentType.isEmpty()) { connection.setRequestProperty("Content-Type", contentType); } } private static URLConnection doGet(String url, Map<String, ? extends Object> headers, String contentType, String charset) throws IOException { URLConnection connection;/* Handle output. */ connection = new URL(url).openConnection(); manageContentTypeHeaders(contentType, charset, connection); manageHeaders(headers, connection); return connection; } private static String extractResponseString(URLConnection connection) throws IOException { /* Handle input. */ HttpURLConnection http = (HttpURLConnection)connection; int status = http.getResponseCode(); String charset = getCharset(connection.getHeaderField("Content-Type")); if (status==200) { return readResponseBody(http, charset); } else { return readErrorResponseBody(http, status, charset); } } private static String readErrorResponseBody(HttpURLConnection http, int status, String charset) { InputStream errorStream = http.getErrorStream(); if ( errorStream!=null ) { String error = charset== null ? read( errorStream ) : read( errorStream, charset ); throw new RuntimeException("STATUS CODE =" + status +" " + error); } else { throw new RuntimeException("STATUS CODE =" + status); } } private static String readResponseBody(HttpURLConnection http, String charset) throws IOException { if (charset != null) { return read(http.getInputStream(), charset); } else { return read(http.getInputStream()); } } private static String getCharset(String contentType) { if (contentType==null) { return null; } String charset = null; for (String param : contentType.replace("","").split(";")) { if (param.startsWith("charset=")) { charset = param.split("=", 2)[1]; break; } } charset = charset == null ? IO.CHARSET : charset; return charset; } |
好吧,你明白了……
以下是测试:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 | static class MyHandler implements HttpHandler { public void handle(HttpExchange t) throws IOException { InputStream requestBody = t.getRequestBody(); String body = IO.read(requestBody); Headers requestHeaders = t.getRequestHeaders(); body = body +" " + copy(requestHeaders).toString(); t.sendResponseHeaders(200, body.length()); OutputStream os = t.getResponseBody(); os.write(body.getBytes()); os.close(); } } @Test public void testHappy() throws Exception { HttpServer server = HttpServer.create(new InetSocketAddress(9212), 0); server.createContext("/test", new MyHandler()); server.setExecutor(null); // creates a default executor server.start(); Thread.sleep(10); Map<String,String> headers = map("foo","bar","fun","sun"); String response = HTTP.postBodyWithContentType("http://localhost:9212/test", headers,"text/plain","hi mom"); System.out.println(response); assertTrue(response.contains("hi mom")); assertTrue(response.contains("Fun=[sun], Foo=[bar]")); response = HTTP.postBodyWithCharset("http://localhost:9212/test", headers,"text/plain","UTF-8","hi mom"); System.out.println(response); assertTrue(response.contains("hi mom")); assertTrue(response.contains("Fun=[sun], Foo=[bar]")); response = HTTP.postBodyWithHeaders("http://localhost:9212/test", headers,"hi mom"); System.out.println(response); assertTrue(response.contains("hi mom")); assertTrue(response.contains("Fun=[sun], Foo=[bar]")); response = HTTP.get("http://localhost:9212/test"); System.out.println(response); response = HTTP.getWithHeaders("http://localhost:9212/test", headers); System.out.println(response); assertTrue(response.contains("Fun=[sun], Foo=[bar]")); response = HTTP.getWithContentType("http://localhost:9212/test", headers,"text/plain"); System.out.println(response); assertTrue(response.contains("Fun=[sun], Foo=[bar]")); response = HTTP.getWithCharSet("http://localhost:9212/test", headers,"text/plain","UTF-8"); System.out.println(response); assertTrue(response.contains("Fun=[sun], Foo=[bar]")); Thread.sleep(10); server.stop(0); } @Test public void testPostBody() throws Exception { HttpServer server = HttpServer.create(new InetSocketAddress(9220), 0); server.createContext("/test", new MyHandler()); server.setExecutor(null); // creates a default executor server.start(); Thread.sleep(10); Map<String,String> headers = map("foo","bar","fun","sun"); String response = HTTP.postBody("http://localhost:9220/test","hi mom"); assertTrue(response.contains("hi mom")); Thread.sleep(10); server.stop(0); } @Test(expected = RuntimeException.class) public void testSad() throws Exception { HttpServer server = HttpServer.create(new InetSocketAddress(9213), 0); server.createContext("/test", new MyHandler()); server.setExecutor(null); // creates a default executor server.start(); Thread.sleep(10); Map<String,String> headers = map("foo","bar","fun","sun"); String response = HTTP.postBodyWithContentType("http://localhost:9213/foo", headers,"text/plain","hi mom"); System.out.println(response); assertTrue(response.contains("hi mom")); assertTrue(response.contains("Fun=[sun], Foo=[bar]")); Thread.sleep(10); server.stop(0); } |
剩下的可以在这里找到:
https://github.com/richardhighttower/boon
我的目标是提供人们希望以一种更简单的方式去做的事情。
起初我被这篇文章误导了,这篇文章偏向于
后来我才意识到,
根据谷歌博客:
Apache HTTP client has fewer bugs on Eclair and Froyo. It is the best choice for these releases. For Gingerbread , HttpURLConnection is the best choice. Its simple API and small size makes it great fit for Android.
Transparent compression and response caching reduce network use, improve speed and save battery. New applications should use HttpURLConnection; it is where we will be spending our energy going forward.
在阅读了这篇文章和其他一些关于流量叠加的问题之后,我确信
一些支持
在android上,不使用urlencodedformentity,使用url编码的表单数据发出post请求
HttpPost在Java项目中工作,而不是在Android中
还有OKHTTP,它是默认情况下高效的HTTP客户机:
- HTTP/2 support allows all requests to the same host to share a socket.
- Connection pooling reduces request latency (if HTTP/2 isn’t available).
- Transparent GZIP shrinks download sizes.
- Response caching avoids the network completely for repeat requests.
首先创建
1 | OkHttpClient client = new OkHttpClient(); |
然后,准备您的
最后,使用
1 | Response response = client.newCall(request).execute(); |
有关更多详细信息,请参阅OKHTTP的文档
您还可以使用jcabi http(我是开发人员)提供的
1 |
查看此博客文章了解更多信息:http://www.yegor256.com/2014/04/11/jcabi-http-intro.html
如果使用HTTP GET,请删除此行
1 | urlConnection.setDoOutput(true); |