Support unbuffered request for uwsgi #431
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
golang test server: package main
import (
"net"
"net/http"
"io"
"os"
"fmt"
"crypto/sha256"
"github.com/mattn/go-uwsgi"
)
var uploaded int64
func handler(w http.ResponseWriter, req *http.Request) {
fmt.Println("New client")
nread := int64(0)
if req.Method == "PUT" {
uploaded = 0
sha := sha256.New()
uri := req.Header.Get("Path_info")
query := req.Header.Get("Query_string")
fmt.Printf("URI: %s ? %s ; content-length: %d\n", uri, query, req.ContentLength)
fmt.Println("doing upload")
buf := make([]byte, 8192)
sz := req.ContentLength
for sz > 0 {
n, err := req.Body.Read(buf)
if err != nil && err != io.EOF {
fmt.Println("Error, not eof:", err)
w.WriteHeader(400)
w.Write([]byte("Error"))
return
}
sha.Write(buf[0:n])
fmt.Println("Read", n, "bytes")
sz -= int64(n)
nread += int64(n)
uploaded += int64(n)
}
fmt.Println("Finished reading, total bytes received", nread)
fmt.Printf("sha256: %x\n", sha.Sum(nil))
w.WriteHeader(200)
w.Write([]byte("OK"));
} else {
// return currently read
w.WriteHeader(200)
w.Write([]byte(fmt.Sprintf("%d", uploaded)));
}
return
}
func main() {
s := "/var/tmp/uwsgitest.sock"
os.Remove(s)
l, e := net.Listen("unix", s)
if e != nil {
fmt.Println("!!! listen failed", e)
return
}
os.Chmod(s, 0666)
http.Serve(&uwsgi.Listener{l}, http.HandlerFunc(handler))
}test client html page <!DOCTYPE html>
<html>
<head>
<script>
function serverstat() {
var stat = document.getElementById('serverstat');
var req = new XMLHttpRequest();
req.onreadystatechange = function() {
if (req.readyState == 4) { // fully received
if (req.status == 200) {
stat.innerHTML = req.responseText;
} else { // error
console.log("GET ERROR:", req.status, req.statusText, req.responseText);
}
}
}
req.open("GET", "/uwsgitest/stat", true);
req.send(null);
}
function doupload() {
var freader = new FileReader();
var xhr = new XMLHttpRequest();
var progspan = document.getElementById('progress');
var file = document.getElementById('filex').files[0];
xhr.upload.addEventListener("progress", function(e) {
if (e.lengthComputable) {
progspan.innerHTML = e.loaded + ', ' + (e.loaded * 100) / e.total + '%';
}
serverstat();
}, false);
xhr.upload.addEventListener("load", function(e) {
console.log("load", e);
}, false);
xhr.upload.addEventListener("error", function(e) {
console.log("error", e);
}, false);
xhr.upload.addEventListener("abort", function(e) {
console.log("abort", e);
}, false);
xhr.open("PUT", "/uwsgitest/uploadfile");
xhr.overrideMimeType("text/plain");
xhr.send(file);
}
</script>
</head>
<body>
<input id="filex" type="file" /><button onclick="doupload()">Upload</button><br><br>
progress: <span id="progress"></span> | server-stat <span id="serverstat"></span>
</body>
</html> |
|
Cool, Thank your for the patch. |
|
|
||
|
|
||
| static ngx_int_t | ||
| ngx_http_uwsgi_output_filter(ngx_http_upstream_output_filter_ctx_t *ctx, |
There was a problem hiding this comment.
Could this work? uwsgi is not http protocol, you should translate the protocol from http to uwsgi.
|
It works. uwsgi is a simple protocol, especially for http connections. tengine/nginx already does the translation from http to uwsgi before it forwards the request through. I ran the upstream server to print out the uwsgi headers received and they appear ok, e.g: In terms of what nginx sends to the upstream server, it first sends the uwsgi header [0, length-of-uwsgi-key-value-pairs, 0]: struct {
uint8 modifier1;
uint16 datasize;
uint8 modifier2;
}Then it sends the HTTP headers as key-value pairs as an array of: struct {
uint16 key_size;
uint8 key[key_size]; /* e.g. CONTENT_TYPE */
uint16 val_size;
uint8 val[val_size];
}Immediately after, nginx forwards the request body as-is, i.e. there is no framing around the request body so it's the responsibility of the backend uwsgi server to process the body up to Content-length. Also, it appears at the moment nginx doesn't support uwsgi keepalive so every request is closed immediately after being processed by the backend. |
|
Leon seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enables uwsgi to use the same style of request_buffering option as fastcgi and proxy upstreams. A chunk of this code was derived from the proxy module. I've tested this with go-uwsgi and an html page that uploads a large file.