关于javascript:通过HTTP将数据从浏览器传输到服务器的方法

Method for streaming data from browser to server via HTTP

是否有任何类似xhr的浏览器API可用于通过HTTP将二进制文件流式传输到服务器?

随着时间的推移,我希望发出一个HTTP PUT请求并以编程方式创建数据。我不想一次创建所有这些数据,因为内存中可能会有它的Gigs。一些pseuudo代码来说明我在做什么:

1
2
3
4
5
6
7
8
9
10
11
12
var dataGenerator = new DataGenerator(); // Generates 8KB UInt8Array every second
var streamToWriteTo;
http.put('/example', function (requestStream) {
  streamToWriteTo = requestStream;
});

dataGenerator.on('data', function (chunk) {
  if (!streamToWriteTo) {
    return;
  }
  streamToWriteTo.write(chunk);
});

我目前已经有了一个Web套接字解决方案,但是为了更好地与一些现有的服务器端代码进行互操作,我更喜欢常规的HTTP。

编辑:我可以使用出血边缘浏览器API。我正在研究fetch API,因为它支持arraybuffer、数据视图、文件等请求主体。如果我能以某种方式伪造出这些对象中的一个,这样我就可以使用带有动态数据的fetch API,这对我来说是可行的。我试图创建一个代理对象,看看是否有任何方法被调用,我可以猴子补丁。不幸的是,浏览器(至少在Chrome中)似乎是在用本机代码而不是在JSLand中进行读取。但是,如果我错了,请纠正我。


我不知道如何使用纯HTML5 API来实现这一点,但一个可能的解决方法是使用Chrome应用程序作为后台服务,为网页提供附加功能。如果您已经愿意使用开发浏览器并启用实验性功能,那么这似乎只是比这更进一步的一步。

Chrome应用程序可以调用chrome.sockets.tcpAPI,在该API上您可以实现任何您想要的协议,包括HTTP和HTTPS。这将提供实现流式处理的灵活性。

只要应用程序声明此用法,常规网页就可以使用chrome.runtimeAPI与应用程序交换消息。这将允许您的网页对您的应用程序进行异步调用。

我写了这个简单的应用程序作为概念证明:

清单.json

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
 "manifest_version" : 2,

 "name" :"Streaming Upload Test",
 "version" :"0.1",

 "app": {
   "background": {
     "scripts": ["background.js"]
    }
  },

 "externally_connectable": {
   "matches": ["*://localhost/*"]
  },

 "sockets": {
   "tcp": {
     "connect":"*:*"
    }
  },

 "permissions": [
  ]
}

背景.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
var mapSocketToPort = {};

chrome.sockets.tcp.onReceive.addListener(function(info) {
  var port = mapSocketToPort[info.socketId];
  port.postMessage(new TextDecoder('utf-8').decode(info.data));
});

chrome.sockets.tcp.onReceiveError.addListener(function(info) {
  chrome.sockets.tcp.close(info.socketId);
  var port = mapSocketToPort[info.socketId];
  port.postMessage();
  port.disconnect();
  delete mapSocketToPort[info.socketId];
});

// Promisify socket API for easier operation sequencing.
// TODO: Check for error and reject.
function socketCreate() {
  return new Promise(function(resolve, reject) {
    chrome.sockets.tcp.create({ persistent: true }, resolve);
  });
}

function socketConnect(s, host, port) {
  return new Promise(function(resolve, reject) {
    chrome.sockets.tcp.connect(s, host, port, resolve);
  });
}

function socketSend(s, data) {
  return new Promise(function(resolve, reject) {
    chrome.sockets.tcp.send(s, data, resolve);
  });
}

chrome.runtime.onConnectExternal.addListener(function(port) {
  port.onMessage.addListener(function(msg) {
    if (!port.state) {
      port.state = msg;

      port.chain = socketCreate().then(function(info) {
        port.socket = info.socketId;
        mapSocketToPort[port.socket] = port;
        return socketConnect(port.socket, 'httpbin.org', 80);
      }).then(function() {
        // TODO: Layer TLS if needed.
      }).then(function() {
        // TODO: Build headers from the request.
        // TODO: Use Transfer-Encoding: chunked.
        var headers =
            'PUT /put HTTP/1.0

'
+
            'Host: httpbin.org

'
+
            'Content-Length: 17

'
+
            '

'
;
        return socketSend(port.socket, new TextEncoder('utf-8').encode(headers).buffer);
      });
    }
    else {
      if (msg) {
        port.chain = port.chain.then(function() {
          // TODO: Use chunked encoding.
          return socketSend(port.socket, new TextEncoder('utf-8').encode(msg).buffer);
        });
      }
    }
  });
});

此应用程序没有用户界面。它监听连接并向http://httpbin.org/put发出硬编码的Put请求(httpbin是一个有用的测试站点,但注意它不支持分块编码)。Put数据(当前硬编码为17个八位字节)从客户机(根据需要使用尽可能少的或尽可能多的消息)流入并发送到服务器。来自服务器的响应被流回到客户机。

这只是概念的证明。真正的应用程序应该:

  • 连接到任何主机和端口。
  • 使用传输编码:分块。
  • 发送流数据结束的信号。
  • 处理套接字错误。
  • 支持TLS(例如,使用Forge)

以下是一个使用应用程序作为服务执行流式上载(17个八位字节)的示例网页(请注意,您必须配置自己的应用程序ID):

1
[cc lang="javascript"]

var my_chrome_app_id='omlafihmmjpklmnlcfkghehxcomggohk';函数streamingupload(url,选项){//打开与Chrome应用程序的连接。参数必须是var port=chrome.runtime.connect(我的chrome-app-id);port.onmessage.addlistener(函数(msg){如果(消息)document.getElementByID("result").textcontent+=msg;其他的port.disconnect();(});//发送参数(必须是JSON可序列化的)。邮件发送端口({网址:url,选项:选项(});//返回要用body数据调用的函数。返回函数(数据){port.postmessage(数据);};}//开始上传。var f=streamingupload('https://httpbin.org/put',方法:'put');//一次传输一个字符的数据。"现在怎么样,棕牛"。拆分("")。前臂(f);

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<P>当我在安装了应用程序的Chrome浏览器中加载此网页时,httpbin返回:</P>[cc lang="javascript"]HTTP/1.1 200 OK
Server: nginx
Date: Sun, 19 Jun 2016 16:54:23 GMT
Content-Type: application/json
Content-Length: 240
Connection: close
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
 "args": {},
 "data":"how now brown cow",
 "files": {},
 "form": {},
 "headers": {
   "Content-Length":"17",
   "Host":"httpbin.org"
  },
 "json": null,
 "origin":"[redacted]",
 "url":"http://httpbin.org/put"
}


我目前正在寻找完全相同的东西(通过Ajax向上游)。我目前所发现的,看起来好像我们在搜索浏览器功能设计的最前沿;-)xmlhttpRequest定义在步骤4 bodyInit中告诉您,此内容提取是(或可以是)可读流。我仍然在搜索(作为一个非WebDeveloper)关于如何创建这样一个东西的信息,并将数据输入到"readablestream"的"另一端"(也就是说应该是一个"writablestream",但我没有找到这一点)。如果你找到了实现这些设计计划的方法,也许你在搜索方面会更好,可以在这里发帖。^ 5斯文


一种利用ReadableStream来传输任意数据的方法;RTCDataChannelUint8Array的形式发送和接收任意数据;TextEncoder创建8000字节的存储在Uint8Array中的随机数据;TextDecoder解码Uint8Array返回给stri的Uint8Array字节。Ng表示,注中也可以使用FileReader.readAsArrayBuffer.readAsText

标记和脚本代码是从MDN - WebRTC: Simple RTCDataChannel sample的示例中修改的,包括adapter.js,其中包含RTCPeerConnection助手;创建您自己的可读流。

另外,当传输的总字节数达到8000 * 8时,示例流被取消:64000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
(function init() {
  var interval, reader, stream, curr, len = 0,
    totalBytes = 8000 * 8,
    data ="abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789",
    randomData = function randomData() {
      var encoder = new TextEncoder();
      var currentStream ="";
      for (var i = 0; i < 8000; i++) {
        currentStream += data[Math.floor(Math.random() * data.length)]
      }
      return encoder.encode(currentStream)
    },
    // optionally reconnect to stream if cancelled
    reconnect = function reconnect() {
      connectButton.disabled = false;
      startup()
    };

  // Define"global" variables

  var connectButton = null;
  var disconnectButton = null;
  var messageInputBox = null;
  var receiveBox = null;

  var localConnection = null; // RTCPeerConnection for our"local" connection
  // adjust this to remote address; or use `ServiceWorker` `onfetch`; other
  var remoteConnection = null; // RTCPeerConnection for the"remote"

  var sendChannel = null; // RTCDataChannel for the local (sender)
  var receiveChannel = null; // RTCDataChannel for the remote (receiver)

  // Functions

  // Set things up, connect event listeners, etc.

  function startup() {
    connectButton = document.getElementById("connectButton");
    disconnectButton = document.getElementById("disconnectButton");
    messageInputBox = document.getElementById("message");
    receiveBox = document.getElementById("receivebox");

    // Set event listeners for user interface widgets

    connectButton.addEventListener("click", connectPeers, false);
    disconnectButton.addEventListener("click", disconnectPeers, false);
  }

  // Connect the two peers. Normally you look for and connect to a remote
  // machine here, but we"re just connecting two local objects, so we can
  // bypass that step.

  function connectPeers() {
    // Create the local connection and its event listeners
    if (len < totalBytes) {
      localConnection = new RTCPeerConnection();

      // Create the data channel and establish its event listeners
      sendChannel = localConnection.createDataChannel("sendChannel");
      sendChannel.onopen = handleSendChannelStatusChange;
      sendChannel.onclose = handleSendChannelStatusChange;

      // Create the remote connection and its event listeners

      remoteConnection = new RTCPeerConnection();
      remoteConnection.ondatachannel = receiveChannelCallback;

      // Set up the ICE candidates for the two peers

      localConnection.onicecandidate = e =>
        !e.candidate || remoteConnection.addIceCandidate(e.candidate)
      .catch(handleAddCandidateError);

      remoteConnection.onicecandidate = e =>
        !e.candidate || localConnection.addIceCandidate(e.candidate)
      .catch(handleAddCandidateError);

      // Now create an offer to connect; this starts the process

      localConnection.createOffer()
      .then(offer => localConnection.setLocalDescription(offer))
      .then(() => remoteConnection
                 .setRemoteDescription(localConnection.localDescription)
       )
      .then(() => remoteConnection.createAnswer())
      .then(answer => remoteConnection
                      .setLocalDescription(answer)
       )
      .then(() => localConnection
                 .setRemoteDescription(remoteConnection.localDescription)
      )
      // start streaming connection
      .then(sendMessage)
      .catch(handleCreateDescriptionError);
    } else {

      alert("total bytes streamed:" + len)
    }

  }

  // Handle errors attempting to create a description;
  // this can happen both when creating an offer and when
  // creating an answer. In this simple example, we handle
  // both the same way.

  function handleCreateDescriptionError(error) {
    console.log("Unable to create an offer:" + error.toString());
  }

  // Handle successful addition of the ICE candidate
  // on the"local" end of the connection.

  function handleLocalAddCandidateSuccess() {
    connectButton.disabled = true;
  }

  // Handle successful addition of the ICE candidate
  // on the"remote" end of the connection.

  function handleRemoteAddCandidateSuccess() {
    disconnectButton.disabled = false;
  }

  // Handle an error that occurs during addition of ICE candidate.

  function handleAddCandidateError() {
    console.log("Oh noes! addICECandidate failed!");
  }

  // Handles clicks on the"Send" button by transmitting
  // a message to the remote peer.

  function sendMessage() {

    stream = new ReadableStream({
      start(controller) {
          interval = setInterval(() => {
            if (sendChannel) {
              curr = randomData();
              len += curr.byteLength;
              // queue current stream
              controller.enqueue([curr, len, sendChannel.send(curr)]);

              if (len >= totalBytes) {
                controller.close();
                clearInterval(interval);
              }
            }
          }, 1000);
        },
        pull(controller) {
          // do stuff during stream
          // call `releaseLock()` if `diconnect` button clicked
          if (!sendChannel) reader.releaseLock();
        },
        cancel(reason) {
          clearInterval(interval);
          console.log(reason);
        }
    });

    reader = stream.getReader({
      mode:"byob"
    });

    reader.read().then(function process(result) {
        if (result.done && len >= totalBytes) {
          console.log("Stream done!");
          connectButton.disabled = false;
          if (len < totalBytes) reconnect();
          return;
        }

        if (!result.done && result.value) {
          var [currentStream, totalStreamLength] = [...result.value];
        }

        if (result.done && len < totalBytes) {
          throw new Error("stream cancelled")
        }

        console.log("currentStream:", currentStream
                   ,"totalStremalength:", totalStreamLength
                   ,"result:", result);
        return reader.read().then(process);
      })
      .catch(function(err) {
        console.log("catch stream cancellation:", err);
        if (len < totalBytes) reconnect()
      });

    reader.closed.then(function() {
      console.log("stream closed")
    })

  }

  // Handle status changes on the local end of the data
  // channel; this is the end doing the sending of data
  // in this example.

  function handleSendChannelStatusChange(event) {
    if (sendChannel) {
      var state = sendChannel.readyState;

      if (state ==="open") {
        disconnectButton.disabled = false;
        connectButton.disabled = true;
      } else {
        connectButton.disabled = false;
        disconnectButton.disabled = true;
      }
    }
  }

  // Called when the connection opens and the data
  // channel is ready to be connected to the remote.

  function receiveChannelCallback(event) {
    receiveChannel = event.channel;
    receiveChannel.onmessage = handleReceiveMessage;
    receiveChannel.onopen = handleReceiveChannelStatusChange;
    receiveChannel.onclose = handleReceiveChannelStatusChange;
  }

  // Handle onmessage events for the receiving channel.
  // These are the data messages sent by the sending channel.

  function handleReceiveMessage(event) {
    var decoder = new TextDecoder();
    var data = decoder.decode(event.data);
    var el = document.createElement("p");
    var txtNode = document.createTextNode(data);

    el.appendChild(txtNode);
    receiveBox.appendChild(el);
  }

  // Handle status changes on the receiver"s channel.

  function handleReceiveChannelStatusChange(event) {
    if (receiveChannel) {
      console.log("Receive channel's status has changed to" +
        receiveChannel.readyState);
    }

    // Here you would do stuff that needs to be done
    // when the channel"s status changes.
  }

  // Close the connection, including data channels if they"re open.
  // Also update the UI to reflect the disconnected status.

  function disconnectPeers() {

    // Close the RTCDataChannels if they"re open.

    sendChannel.close();
    receiveChannel.close();

    // Close the RTCPeerConnections

    localConnection.close();
    remoteConnection.close();

    sendChannel = null;
    receiveChannel = null;
    localConnection = null;
    remoteConnection = null;

    // Update user interface elements


    disconnectButton.disabled = true;
    // cancel stream on `click` of `disconnect` button,
    // pass `reason` for cancellation as parameter
    reader.cancel("stream cancelled");
  }

  // Set up an event listener which will run the startup
  // function once the page is done loading.

  window.addEventListener("load", startup, false);
})();

plnkr http://plnkr.co/edit/cln6uxgmzwe2eqcfnxfo?P=预览


您可以使用PromisesetTimeout,递归。另请参见放置vs post in rest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
var count = 0, total = 0, timer = null, d = 500, stop = false, p = void 0
, request = function request () {
              return new XMLHttpRequest()
            };
function sendData() {
  p = Promise.resolve(generateSomeBinaryData()).then(function(data) {
    var currentRequest = request();
    currentRequest.open("POST","http://example.com");
    currentRequest.onload = function () {
      ++count; // increment `count`
      total += data.byteLength; // increment total bytes posted to server
    }

    currentRequest.onloadend = function () {
      if (stop) { // stop recursion
        throw new Error("aborted") // `throw` error to `.catch()`
      } else {
        timer = setTimeout(sendData, d); // recursively call `sendData`
      }
    }
    currentRequest.send(data); // `data`: `Uint8Array`; `TypedArray`
    return currentRequest; // return `currentRequest`
  });
  return p // return `Promise` : `p`
}

var curr = sendData();

curr.then(function(current) {
  console.log(current) // current post request
})
.catch(function(err) {
  console.log(e) // handle aborted `request`; errors
});


服务器发送的事件和WebSockets是首选的方法,但在您的情况下,您希望创建一个表示状态传输、REST、API并使用长轮询。看看如何实现基本的"长轮询"?

在客户端和服务器端都处理长轮询过程。必须将服务器脚本和HTTP服务器配置为支持长轮询。

除了长轮询之外,短轮询(xhr/ajax)需要浏览器轮询服务器。