Method for streaming data from browser to server via HTTP
是否有任何类似xhr的浏览器API可用于通过HTTP将二进制文件流式传输到服务器?
随着时间的推移,我希望发出一个HTTP PUT请求并以编程方式创建数据。我不想一次创建所有这些数据,因为内存中可能会有它的Gigs。一些pseuudo代码来说明我在做什么:
1 2 3 4 5 6 7 8 9 10 11 12 | var dataGenerator = new DataGenerator(); // Generates 8KB UInt8Array every second var streamToWriteTo; http.put('/example', function (requestStream) { streamToWriteTo = requestStream; }); dataGenerator.on('data', function (chunk) { if (!streamToWriteTo) { return; } streamToWriteTo.write(chunk); }); |
我目前已经有了一个Web套接字解决方案,但是为了更好地与一些现有的服务器端代码进行互操作,我更喜欢常规的HTTP。
编辑:我可以使用出血边缘浏览器API。我正在研究fetch API,因为它支持arraybuffer、数据视图、文件等请求主体。如果我能以某种方式伪造出这些对象中的一个,这样我就可以使用带有动态数据的fetch API,这对我来说是可行的。我试图创建一个代理对象,看看是否有任何方法被调用,我可以猴子补丁。不幸的是,浏览器(至少在Chrome中)似乎是在用本机代码而不是在JSLand中进行读取。但是,如果我错了,请纠正我。
我不知道如何使用纯HTML5 API来实现这一点,但一个可能的解决方法是使用Chrome应用程序作为后台服务,为网页提供附加功能。如果您已经愿意使用开发浏览器并启用实验性功能,那么这似乎只是比这更进一步的一步。
Chrome应用程序可以调用
只要应用程序声明此用法,常规网页就可以使用
我写了这个简单的应用程序作为概念证明:
清单.json
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | { "manifest_version" : 2, "name" :"Streaming Upload Test", "version" :"0.1", "app": { "background": { "scripts": ["background.js"] } }, "externally_connectable": { "matches": ["*://localhost/*"] }, "sockets": { "tcp": { "connect":"*:*" } }, "permissions": [ ] } |
背景.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | var mapSocketToPort = {}; chrome.sockets.tcp.onReceive.addListener(function(info) { var port = mapSocketToPort[info.socketId]; port.postMessage(new TextDecoder('utf-8').decode(info.data)); }); chrome.sockets.tcp.onReceiveError.addListener(function(info) { chrome.sockets.tcp.close(info.socketId); var port = mapSocketToPort[info.socketId]; port.postMessage(); port.disconnect(); delete mapSocketToPort[info.socketId]; }); // Promisify socket API for easier operation sequencing. // TODO: Check for error and reject. function socketCreate() { return new Promise(function(resolve, reject) { chrome.sockets.tcp.create({ persistent: true }, resolve); }); } function socketConnect(s, host, port) { return new Promise(function(resolve, reject) { chrome.sockets.tcp.connect(s, host, port, resolve); }); } function socketSend(s, data) { return new Promise(function(resolve, reject) { chrome.sockets.tcp.send(s, data, resolve); }); } chrome.runtime.onConnectExternal.addListener(function(port) { port.onMessage.addListener(function(msg) { if (!port.state) { port.state = msg; port.chain = socketCreate().then(function(info) { port.socket = info.socketId; mapSocketToPort[port.socket] = port; return socketConnect(port.socket, 'httpbin.org', 80); }).then(function() { // TODO: Layer TLS if needed. }).then(function() { // TODO: Build headers from the request. // TODO: Use Transfer-Encoding: chunked. var headers = 'PUT /put HTTP/1.0 ' + 'Host: httpbin.org ' + 'Content-Length: 17 ' + ' '; return socketSend(port.socket, new TextEncoder('utf-8').encode(headers).buffer); }); } else { if (msg) { port.chain = port.chain.then(function() { // TODO: Use chunked encoding. return socketSend(port.socket, new TextEncoder('utf-8').encode(msg).buffer); }); } } }); }); |
。
此应用程序没有用户界面。它监听连接并向
这只是概念的证明。真正的应用程序应该:
- 连接到任何主机和端口。
- 使用传输编码:分块。
- 发送流数据结束的信号。
- 处理套接字错误。
- 支持TLS(例如,使用Forge)
以下是一个使用应用程序作为服务执行流式上载(17个八位字节)的示例网页(请注意,您必须配置自己的应用程序ID):
1 | [cc lang="javascript"] |
var my_chrome_app_id='omlafihmmjpklmnlcfkghehxcomggohk';函数streamingupload(url,选项){//打开与Chrome应用程序的连接。参数必须是var port=chrome.runtime.connect(我的chrome-app-id);port.onmessage.addlistener(函数(msg){如果(消息)document.getElementByID("result").textcontent+=msg;其他的port.disconnect();(});//发送参数(必须是JSON可序列化的)。邮件发送端口({网址:url,选项:选项(});//返回要用body数据调用的函数。返回函数(数据){port.postmessage(数据);};}//开始上传。var f=streamingupload('https://httpbin.org/put',方法:'put');//一次传输一个字符的数据。"现在怎么样,棕牛"。拆分("")。前臂(f);
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | <P>当我在安装了应用程序的Chrome浏览器中加载此网页时,httpbin返回:</P>[cc lang="javascript"]HTTP/1.1 200 OK Server: nginx Date: Sun, 19 Jun 2016 16:54:23 GMT Content-Type: application/json Content-Length: 240 Connection: close Access-Control-Allow-Origin: * Access-Control-Allow-Credentials: true { "args": {}, "data":"how now brown cow", "files": {}, "form": {}, "headers": { "Content-Length":"17", "Host":"httpbin.org" }, "json": null, "origin":"[redacted]", "url":"http://httpbin.org/put" } |
。
我目前正在寻找完全相同的东西(通过Ajax向上游)。我目前所发现的,看起来好像我们在搜索浏览器功能设计的最前沿;-)xmlhttpRequest定义在步骤4 bodyInit中告诉您,此内容提取是(或可以是)可读流。我仍然在搜索(作为一个非WebDeveloper)关于如何创建这样一个东西的信息,并将数据输入到"readablestream"的"另一端"(也就是说应该是一个"writablestream",但我没有找到这一点)。如果你找到了实现这些设计计划的方法,也许你在搜索方面会更好,可以在这里发帖。^ 5斯文
一种利用
标记和脚本代码是从
另外,当传输的总字节数达到
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 | (function init() { var interval, reader, stream, curr, len = 0, totalBytes = 8000 * 8, data ="abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789", randomData = function randomData() { var encoder = new TextEncoder(); var currentStream =""; for (var i = 0; i < 8000; i++) { currentStream += data[Math.floor(Math.random() * data.length)] } return encoder.encode(currentStream) }, // optionally reconnect to stream if cancelled reconnect = function reconnect() { connectButton.disabled = false; startup() }; // Define"global" variables var connectButton = null; var disconnectButton = null; var messageInputBox = null; var receiveBox = null; var localConnection = null; // RTCPeerConnection for our"local" connection // adjust this to remote address; or use `ServiceWorker` `onfetch`; other var remoteConnection = null; // RTCPeerConnection for the"remote" var sendChannel = null; // RTCDataChannel for the local (sender) var receiveChannel = null; // RTCDataChannel for the remote (receiver) // Functions // Set things up, connect event listeners, etc. function startup() { connectButton = document.getElementById("connectButton"); disconnectButton = document.getElementById("disconnectButton"); messageInputBox = document.getElementById("message"); receiveBox = document.getElementById("receivebox"); // Set event listeners for user interface widgets connectButton.addEventListener("click", connectPeers, false); disconnectButton.addEventListener("click", disconnectPeers, false); } // Connect the two peers. Normally you look for and connect to a remote // machine here, but we"re just connecting two local objects, so we can // bypass that step. function connectPeers() { // Create the local connection and its event listeners if (len < totalBytes) { localConnection = new RTCPeerConnection(); // Create the data channel and establish its event listeners sendChannel = localConnection.createDataChannel("sendChannel"); sendChannel.onopen = handleSendChannelStatusChange; sendChannel.onclose = handleSendChannelStatusChange; // Create the remote connection and its event listeners remoteConnection = new RTCPeerConnection(); remoteConnection.ondatachannel = receiveChannelCallback; // Set up the ICE candidates for the two peers localConnection.onicecandidate = e => !e.candidate || remoteConnection.addIceCandidate(e.candidate) .catch(handleAddCandidateError); remoteConnection.onicecandidate = e => !e.candidate || localConnection.addIceCandidate(e.candidate) .catch(handleAddCandidateError); // Now create an offer to connect; this starts the process localConnection.createOffer() .then(offer => localConnection.setLocalDescription(offer)) .then(() => remoteConnection .setRemoteDescription(localConnection.localDescription) ) .then(() => remoteConnection.createAnswer()) .then(answer => remoteConnection .setLocalDescription(answer) ) .then(() => localConnection .setRemoteDescription(remoteConnection.localDescription) ) // start streaming connection .then(sendMessage) .catch(handleCreateDescriptionError); } else { alert("total bytes streamed:" + len) } } // Handle errors attempting to create a description; // this can happen both when creating an offer and when // creating an answer. In this simple example, we handle // both the same way. function handleCreateDescriptionError(error) { console.log("Unable to create an offer:" + error.toString()); } // Handle successful addition of the ICE candidate // on the"local" end of the connection. function handleLocalAddCandidateSuccess() { connectButton.disabled = true; } // Handle successful addition of the ICE candidate // on the"remote" end of the connection. function handleRemoteAddCandidateSuccess() { disconnectButton.disabled = false; } // Handle an error that occurs during addition of ICE candidate. function handleAddCandidateError() { console.log("Oh noes! addICECandidate failed!"); } // Handles clicks on the"Send" button by transmitting // a message to the remote peer. function sendMessage() { stream = new ReadableStream({ start(controller) { interval = setInterval(() => { if (sendChannel) { curr = randomData(); len += curr.byteLength; // queue current stream controller.enqueue([curr, len, sendChannel.send(curr)]); if (len >= totalBytes) { controller.close(); clearInterval(interval); } } }, 1000); }, pull(controller) { // do stuff during stream // call `releaseLock()` if `diconnect` button clicked if (!sendChannel) reader.releaseLock(); }, cancel(reason) { clearInterval(interval); console.log(reason); } }); reader = stream.getReader({ mode:"byob" }); reader.read().then(function process(result) { if (result.done && len >= totalBytes) { console.log("Stream done!"); connectButton.disabled = false; if (len < totalBytes) reconnect(); return; } if (!result.done && result.value) { var [currentStream, totalStreamLength] = [...result.value]; } if (result.done && len < totalBytes) { throw new Error("stream cancelled") } console.log("currentStream:", currentStream ,"totalStremalength:", totalStreamLength ,"result:", result); return reader.read().then(process); }) .catch(function(err) { console.log("catch stream cancellation:", err); if (len < totalBytes) reconnect() }); reader.closed.then(function() { console.log("stream closed") }) } // Handle status changes on the local end of the data // channel; this is the end doing the sending of data // in this example. function handleSendChannelStatusChange(event) { if (sendChannel) { var state = sendChannel.readyState; if (state ==="open") { disconnectButton.disabled = false; connectButton.disabled = true; } else { connectButton.disabled = false; disconnectButton.disabled = true; } } } // Called when the connection opens and the data // channel is ready to be connected to the remote. function receiveChannelCallback(event) { receiveChannel = event.channel; receiveChannel.onmessage = handleReceiveMessage; receiveChannel.onopen = handleReceiveChannelStatusChange; receiveChannel.onclose = handleReceiveChannelStatusChange; } // Handle onmessage events for the receiving channel. // These are the data messages sent by the sending channel. function handleReceiveMessage(event) { var decoder = new TextDecoder(); var data = decoder.decode(event.data); var el = document.createElement("p"); var txtNode = document.createTextNode(data); el.appendChild(txtNode); receiveBox.appendChild(el); } // Handle status changes on the receiver"s channel. function handleReceiveChannelStatusChange(event) { if (receiveChannel) { console.log("Receive channel's status has changed to" + receiveChannel.readyState); } // Here you would do stuff that needs to be done // when the channel"s status changes. } // Close the connection, including data channels if they"re open. // Also update the UI to reflect the disconnected status. function disconnectPeers() { // Close the RTCDataChannels if they"re open. sendChannel.close(); receiveChannel.close(); // Close the RTCPeerConnections localConnection.close(); remoteConnection.close(); sendChannel = null; receiveChannel = null; localConnection = null; remoteConnection = null; // Update user interface elements disconnectButton.disabled = true; // cancel stream on `click` of `disconnect` button, // pass `reason` for cancellation as parameter reader.cancel("stream cancelled"); } // Set up an event listener which will run the startup // function once the page is done loading. window.addEventListener("load", startup, false); })(); |
号
plnkr http://plnkr.co/edit/cln6uxgmzwe2eqcfnxfo?P=预览
您可以使用
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | var count = 0, total = 0, timer = null, d = 500, stop = false, p = void 0 , request = function request () { return new XMLHttpRequest() }; function sendData() { p = Promise.resolve(generateSomeBinaryData()).then(function(data) { var currentRequest = request(); currentRequest.open("POST","http://example.com"); currentRequest.onload = function () { ++count; // increment `count` total += data.byteLength; // increment total bytes posted to server } currentRequest.onloadend = function () { if (stop) { // stop recursion throw new Error("aborted") // `throw` error to `.catch()` } else { timer = setTimeout(sendData, d); // recursively call `sendData` } } currentRequest.send(data); // `data`: `Uint8Array`; `TypedArray` return currentRequest; // return `currentRequest` }); return p // return `Promise` : `p` } var curr = sendData(); curr.then(function(current) { console.log(current) // current post request }) .catch(function(err) { console.log(e) // handle aborted `request`; errors }); |
服务器发送的事件和WebSockets是首选的方法,但在您的情况下,您希望创建一个表示状态传输、REST、API并使用长轮询。看看如何实现基本的"长轮询"?
在客户端和服务器端都处理长轮询过程。必须将服务器脚本和HTTP服务器配置为支持长轮询。
除了长轮询之外,短轮询(xhr/ajax)需要浏览器轮询服务器。