Initial Commit

Fork of lhttpc.

.gitignore

@@ -0,0 +1,20 @@
+/ebin
+/deps
+/logs
+/log
+.DS_Store
+/test/*.beam
+/test/*.log
+*.swp
+*.swo
+erl_crash.dump
+/config/production/*.bert
+.eunit
+
+doc/edoc-info
+doc/*.html
+doc/*.png
+doc/*.css
+util/*.beam
+cover_report/
+

CHANGELOG

@@ -0,0 +1,77 @@
+Version ?:
+* Add support for connect_options
+    * Enables the user to pass socket options, for instance ip and port,
+      that will be used when connecting the socket
+    * Allows the user to specify SSL options during the connect phase
+* Add support for "streaming" of entities
+* Add start/0 and stop/0
+* Fix for unexpected messages after request has been completed
+    * When the client process is trapping exits (which some eunit versions seem
+      to be doing by default) there would be {'EXIT', Pid, normal} messages left
+      after completing requests which came from the lhttpc_client process.
+      These are now avoided.
+* Add rebar support (thanks to Benoit Chesneau)
+
+Version 1.2.5:
+* Fix for decoding chunked HTTP responses with extraneous whitespace
+  (thanks to Bob Ippolito)
+    * api.facebook.com includes extra whitespace in its chunked HTTP response
+      length, e.g. "6  \r\n" instead of "6\r\n". 
+
+Version 1.2.4:
+* Fix handling of HTTP response code 304 (patch by Ben Slavin)
+
+Version 1.2.3:
+* Fix handling of unexpected 100 Continue response (patch by Magnus Henoch)
+
+Version 1.2.2:
+* Fix Host header, add port number (reported by Benoit Chesneau)
+
+Version 1.2.1:
+* Fix confusion of arguments in request/9 (introduced in 1.2.0)
+
+Version 1.2.0:
+* Add support for options
+    * {connect_timeout, Milliseconds}: Aborts the connect phase after
+      Milliseconds
+    * {send_retry, N}: The client will retry sending the request N times
+      if the connection is closed
+* Add support for downloading chunked transfer encoding
+* More consistent error handling
+    * The client can either return {ok, Result} or {error, Reason}. Reason
+      (which wasn't very well defined before) can now be:
+        * connection_closed: The server closed the connection on us (N times
+          in case it happens just after sending the request) 
+        * connect_timeout: If the TCP stack gives up or we hit the
+          connect_timeout option
+        * timeout: If the overall request timeout value is hit
+    * All other errors (socket, protocol etc. will result in a runtime error)
+* Better connection handling (issues #2 and #3 on bitbucket.org/etc/lhttpc)
+    * Now understands what to do with connections to servers < HTTP/1.1
+    * Now respects "Connection: close" in request headers, which means
+      clients can choose to not use persistent connections
+* RFC Compliance
+    * Fix reading of chunked encoding since section 3.6 claims that "All
+      transfer-coding values are case-insensitive"
+    * Support for responses that doesn't have an entity body (thanks to Steve
+      Ellis)
+        * No body for the HEAD and OPTIONS (unless indicated for the latter)
+          methods
+        * Don't try to read a body when receiving a 204 status
+
+Version 1.1.2:
+* Fix minor error with {active, once} instead of {active, true} in manager
+* Remove socket and try to find another if the manager can't set controlling
+  process due to socket error
+* Improve test suite 
+
+Version 1.1.1:
+* Fix problem with empty lists in dicts when last socket is consumed
+* Improve test suite to include cover report 
+
+Version 1.1.0:
+* Support for configurable connection timeout 
+
+Version 1.0.0:
+* Initial version
+* Persistent connections have hardcoded timeout

LICENCE

@@ -0,0 +1,23 @@
+Copyright (c) 2009, Erlang Training and Consulting Ltd.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+   * Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+   * Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in the
+     documentation and/or other materials provided with the distribution.
+   * Neither the name of Erlang Training and Consulting Ltd. nor the
+     names of its contributors may be used to endorse or promote products
+     derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY Erlang Training and Consulting Ltd. ''AS IS''
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL Erlang Training and Consulting Ltd. BE
+LIABLE SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Makefile

@@ -0,0 +1,23 @@
+REBAR := ./rebar
+
+.PHONY: all doc clean test dialyzer
+
+all: compile doc
+
+compile:
+	$(REBAR) get-deps compile
+
+doc:
+	$(REBAR) doc skip_deps=true
+
+test:
+	$(REBAR) eunit skip_deps=true
+
+dialyzer:
+	$(REBAR) analyze
+
+release: all dialyzer test
+	$(REBAR) release
+
+clean:
+	$(REBAR) clean

README

@@ -0,0 +1,15 @@
+Dependencies:
+ * Erlang/OTP R13-B or newer
+   * Application compiler to build, kernel, stdlib and ssl to run
+
+Building: 
+For versions > 1.2.5, lhttpc is built using rebar. Take a look at http://bitbucket.org/basho/rebar/wiki/Home for more information. There is still a Makefile with some of the old make targets, such as all, doc, test etc. for those who prefer that. The makefile will however just call rebar.
+
+Configuration: (environment variables)
+ * connection_timeout: The time (in milliseconds) the client will try to
+                       kepp a HTTP/1.1 connection open. Changing this value
+                       in runtime has no effect, this can however be done
+                       through lhttpc_manager:update_connection_timeout/1.
+
+NOTE: THIS FORK OF LHTTPC IS ONLY RECOMMENDED IF YOU HAVE MANY REQUESTS TO DO TO A FEW RESTRICTED DOMAINS.
+It contains load-balancing mechanisms described in http://ferd.ca/rtb-where-erlang-blooms.html. It is not meant for general purpose use.

TODO

@@ -0,0 +1,3 @@
+* Add support for HTTP redirects
+* Add proper support for Expect: 100-continue
+* Implement semaphore to enable connection count limits

doc/overview.edoc

@@ -0,0 +1,11 @@
+@author Oscar Hellstr�m <[email protected]>
+@doc A lightweight HTTP client.
+The only functions of much interest right now are {@link
+lhttpc:request/4}, {@link lhttpc:request/5}, {@link lhttpc:request/6} and
+{@link lhttpc:request/9}.
+
+<h1>Configuration</h1>
+<h2>`connection_timeout'</h2>
+The maximum time (in milliseconds) the client will keep a TCP connection
+open to a server.
+@end

include/dlhttpc_types.hrl

@@ -0,0 +1,49 @@
+%%% ----------------------------------------------------------------------------
+%%% Copyright (c) 2009, Erlang Training and Consulting Ltd.
+%%% All rights reserved.
+%%% 
+%%% Redistribution and use in source and binary forms, with or without
+%%% modification, are permitted provided that the following conditions are met:
+%%%    * Redistributions of source code must retain the above copyright
+%%%      notice, this list of conditions and the following disclaimer.
+%%%    * Redistributions in binary form must reproduce the above copyright
+%%%      notice, this list of conditions and the following disclaimer in the
+%%%      documentation and/or other materials provided with the distribution.
+%%%    * Neither the name of Erlang Training and Consulting Ltd. nor the
+%%%      names of its contributors may be used to endorse or promote products
+%%%      derived from this software without specific prior written permission.
+%%% 
+%%% THIS SOFTWARE IS PROVIDED BY Erlang Training and Consulting Ltd. ''AS IS''
+%%% AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+%%% IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+%%% ARE DISCLAIMED. IN NO EVENT SHALL Erlang Training and Consulting Ltd. BE
+%%% LIABLE SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+%%% BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+%%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+%%% OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+%%% ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+%%% ----------------------------------------------------------------------------
+
+-type header() :: {string() | atom(), string()}.
+-type headers() :: [header()].
+
+-type socket() :: _.
+
+-type option() ::
+        {connect_options, [term()]} |
+        {connect_timeout, timeout()} |
+        {connection_timeout, timeout()} |
+        {max_connections, pos_integer()} |
+        {stream_to, pid()} |
+        {send_retry, non_neg_integer()} |
+        {partial_upload, non_neg_integer() | infinity} |
+        {partial_download, pid(), non_neg_integer() | infinity}.
+
+-type options() :: [option()].
+
+-type host() :: string() | {integer(), integer(), integer(), integer()}.
+
+-type socket_options() :: [{atom(), term()} | atom()].
+
+-type window_size() :: non_neg_integer() | infinity.
+

rebar.config

@@ -0,0 +1,7 @@
+{erl_opts, [debug_info]}.
+{cover_enabled, true}.
+
+{deps, [
+  {dispcount, "0.1.0",
+    {git, "https://github.com/ferd/dispcount.git", "v0.1.0"}}
+]}.

src/dlhttpc.app.src

@@ -0,0 +1,14 @@
+{application, dlhttpc,
+ [{description, "Dispcount + Lightweight HTTP Client"},
+  {vsn, "0.1.0"},
+  {modules, [dlhttpc,dlhttpc_sup,dlhttpc_client,dlhttpc_sock,lhttp_lb]},
+  {registered, [dlhttpc_sup]},
+  {applications, [kernel, stdlib, ssl, crypto, dispcount]},
+  {mod, {dlhttpc, []}},
+  {env, [{connection_timeout, 60000},
+         {restart, permanent},
+         {shutdown, 10000},
+         {maxr, 10},
+         {maxt, 1}]}
+ ]}.
+