Emacs 25.1 error solved: url-http-create-request: Multibyte text in HTTP request

Emacs 25.1 error solved: url-http-create-request: Multibyte text in HTTP request

After update to 25.1, I found that I can't use url-http to request any more. Then I googled for hours, but not found a good solution. So I update the emacs source code from github, and build a latest w64 version 26.0.50, but the error is still existed.

Today, I decide to spend some time to read the code. Then I find this in file:

c:/emacs/share/emacs/26.0.50/lisp/url/url-http.el.gz

line: 395

;; Bug#23750
(unless (= (string-bytes request)
           (length request))
  (error "Multibyte text in HTTP request: %s" request))

The error occurs when our content contains multibyte, then (string-bytes request) is not equal to (length request) .

And I checked the comment (;; Bug#23750) above, which leads a bug to json-mode. So simple delete this code can solve our problem, but that can lead to Bug#23750 again. So we should make sure that our request data get encoded. A simple way is to modified the code as follows:

;; Bug#23750
(setq request (url-http--encode-string request))
(unless (= (string-bytes request)
           (length request))
  (error "Multibyte text in HTTP request: %s" request))

The whole function:

(defun url-http-create-request (&optional ref-url)
  "Create an HTTP request for `url-http-target-url', referred to by REF-URL."
  (let* ((extra-headers)
         (request nil)
         (no-cache (cdr-safe (assoc "Pragma" url-http-extra-headers)))
         (using-proxy url-http-proxy)
         (proxy-auth (if (or (cdr-safe (assoc "Proxy-Authorization"
                                              url-http-extra-headers))
                             (not using-proxy))
                         nil
                       (let ((url-basic-auth-storage
                              'url-http-proxy-basic-auth-storage))
                         (url-get-authentication url-http-proxy nil 'any nil))))
         (real-fname (url-filename url-http-target-url))
         (host (url-http--encode-string (url-host url-http-target-url)))
         (auth (if (cdr-safe (assoc "Authorization" url-http-extra-headers))
                   nil
                 (url-get-authentication (or
                                          (and (boundp 'proxy-info)
                                               proxy-info)
                                          url-http-target-url) nil 'any nil))))
    (if (equal "" real-fname)
        (setq real-fname "/"))
    (setq no-cache (and no-cache (string-match "no-cache" no-cache)))
    (if auth
        (setq auth (concat "Authorization: " auth "
")))
    (if proxy-auth
        (setq proxy-auth (concat "Proxy-Authorization: " proxy-auth "
")))

    ;; Protection against stupid values in the referrer
    (if (and ref-url (stringp ref-url) (or (string= ref-url "file:nil")
                                           (string= ref-url "")))
        (setq ref-url nil))

    ;; We do not want to expose the referrer if the user is paranoid.
    (if (or (memq url-privacy-level '(low high paranoid))
            (and (listp url-privacy-level)
                 (memq 'lastloc url-privacy-level)))
        (setq ref-url nil))

    ;; url-http-extra-headers contains an assoc-list of
    ;; header/value pairs that we need to put into the request.
    (setq extra-headers (mapconcat
                         (lambda (x)
                           (concat (car x) ": " (cdr x)))
                         url-http-extra-headers "
"))
    (if (not (equal extra-headers ""))
        (setq extra-headers (concat extra-headers "
")))

    ;; This was done with a call to `format'.  Concatenating parts has
    ;; the advantage of keeping the parts of each header together and
    ;; allows us to elide null lines directly, at the cost of making
    ;; the layout less clear.
    (setq request
          (concat
             ;; The request
             (or url-http-method "GET") " "
             (url-http--encode-string
              (if using-proxy (url-recreate-url url-http-target-url) real-fname))
             " HTTP/" url-http-version "
"
             ;; Version of MIME we speak
             "MIME-Version: 1.0
"
             ;; (maybe) Try to keep the connection open
             "Connection: " (if (or using-proxy
                                    (not url-http-attempt-keepalives))
                                "close" "keep-alive") "
"
                                ;; HTTP extensions we support
             (if url-extensions-header
                 (format
                  "Extension: %s
" url-extensions-header))
             ;; Who we want to talk to
             (if (/= (url-port url-http-target-url)
                     (url-scheme-get-property
                      (url-type url-http-target-url) 'default-port))
                 (format
                  "Host: %s:%d
" (puny-encode-domain host)
                  (url-port url-http-target-url))
               (format "Host: %s
" (puny-encode-domain host)))
             ;; Who its from
             (if url-personal-mail-address
                 (concat
                  "From: " url-personal-mail-address "
"))
             ;; Encodings we understand
             (if (or url-mime-encoding-string
                     ;; MS-Windows loads zlib dynamically, so recheck
                     ;; in case they made it available since
                     ;; initialization in url-vars.el.
                     (and (eq 'system-type 'windows-nt)
                          (fboundp 'zlib-available-p)
                          (zlib-available-p)
                          (setq url-mime-encoding-string "gzip")))
                 (concat
                  "Accept-encoding: " url-mime-encoding-string "
"))
             (if url-mime-charset-string
                 (concat
                  "Accept-charset: "
                  (url-http--encode-string url-mime-charset-string)
                  "
"))
             ;; Languages we understand
             (if url-mime-language-string
                 (concat
                  "Accept-language: " url-mime-language-string "
"))
             ;; Types we understand
             "Accept: " (or url-mime-accept-string "*/*") "
"
             ;; User agent
             (url-http-user-agent-string)
             ;; Proxy Authorization
             proxy-auth
             ;; Authorization
             auth
             ;; Cookies
             (when (url-use-cookies url-http-target-url)
               (url-http--encode-string
                (url-cookie-generate-header-lines
                 host real-fname
                 (equal "https" (url-type url-http-target-url)))))
             ;; If-modified-since
             (if (and (not no-cache)
                      (member url-http-method '("GET" nil)))
                 (let ((tm (url-is-cached url-http-target-url)))
                   (if tm
                       (concat "If-modified-since: "
                               (url-get-normalized-date tm) "
"))))
             ;; Whence we came
             (if ref-url (concat
                          "Referer: " ref-url "
"))
             extra-headers
             ;; Length of data
             (if url-http-data
                 (concat
                  "Content-length: " (number-to-string
                                      (length url-http-data))
                  "
"))
             ;; End request
             "
"
             ;; Any data
             url-http-data))
    ;; Bug#23750
    (setq request (url-http--encode-string request))
    (unless (= (string-bytes request)
               (length request))
      (error "Multibyte text in HTTP request: %s" request))
    (url-http-debug "Request is: 
%s" request)
    request))

Add this to the configuration file. Then we can request multibyte content like Chinese and Japanese.

This is my Emacs configuration package (contains a Cnblogs package to write cnblogs blogs, download from cnblogs, and fix some bugs):

https://github.com/yangwen0228/unimacs

Date: 2016-12-30 21:14

Created: 2017-01-15 周日 16:09

Validate