Merge branch 'master' of github.com:HelloZeroNet/ZeroNet

README.md

@@ -1,22 +1,34 @@
 # ZeroNet
 
-Decentralized websites using Bitcoin crypto and BitTorrent network
+Decentralized websites using Bitcoin crypto and the BitTorrent network
 
 
 ## Why?
- - We believe in open, free and uncensored network and communication.
- - No single point of failure: Site goes on until at least 1 peer serving it.
- - No hosting costs: Site served by visitors.
- - Impossible to shut down: It's nowhere because it's everywhere.
- - Fast and works offline: You can access the site even if your internet is gone.
+
+* We believe in open, free, and uncensored network and communication.
+* No single point of failure: Site remains online so long as at least 1 peer
+  serving it.
+* No hosting costs: Sites are served by visitors.
+* Impossible to shut down: It's nowhere because it's everywhere.
+* Fast and works offline: You can access the site even if your internet is
+  unavailable.
 
 
 ## How does it work?
- - After starting `zeronet.py` you will be able to visit zeronet sites using http://127.0.0.1:43110/{zeronet_address} (eg. http://127.0.0.1:43110/1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr).
- - When you visit a new zeronet site, it's trying to find peers using BitTorrent network and download the site files (html, css, js...) from them.
- - Each visited sites become also served by You.
- - Every site containing a `site.json` which holds all other files sha512 hash and a sign generated using site's private key.
- - If the site owner (who has the private key for the site address) modifies the site, then he/she signs the new `content.json` and publish it to the peers. After the peers have verified the `content.json` integrity (using the sign), they download the modified files and publish the new content to other peers.
+
+* After starting `zeronet.py` you will be able to visit zeronet sites using
+  `http://127.0.0.1:43110/{zeronet_address}` (eg. 
+  `http://127.0.0.1:43110/1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr`).
+* When you visit a new zeronet site, it tries to find peers using the BitTorrent
+  network so it can download the site files (html, css, js...) from them.
+* Each visited site becomes also served by you.
+* Every site contains a `site.json` which holds all other files in a sha512 hash
+  and a signature generated using site's private key.
+* If the site owner (who has the private key for the site address) modifies the
+  site, then he/she signs the new `content.json` and publishes it to the peers.
+  After the peers have verified the `content.json` integrity (using the
+  signature), they download the modified files and publish the new content to
+  other peers.
 
 
 ## Screenshot
@@ -25,39 +37,45 @@ Decentralized websites using Bitcoin crypto and BitTorrent network
 
 
 ## How to join?
-Windows:
- - [Install Python 2.7](https://www.python.org/ftp/python/2.7.9/python-2.7.9.msi)
- - [Install Python ZeroMQ](http://zeronet.io/files/windows/pyzmq-14.4.1.win32-py2.7.exe)
- - [Install Python Greenlet](http://zeronet.io/files/windows/greenlet-0.4.5.win32-py2.7.exe)
- - [Install Python Gevent](http://zeronet.io/files/windows/gevent-1.0.1.win32-py2.7.exe)
- - [Install Python MsgPack](http://zeronet.io/files/windows/msgpack-python-0.4.2.win32-py2.7.exe)
- - start `start.py`
-
-Linux (Debian):
- - `apt-get install python-pip` 
- - `pip install pyzmq` (if it drops a compile error then run `apt-get install python-dev` and try again) 
- - `pip install gevent`
- - `pip install msgpack-python`
- - start using `python zeronet.py`
-
-Linux (Without root access):
- - `wget https://bootstrap.pypa.io/get-pip.py` 
- - `python get-pip.py --user pyzmq gevent msgpack-python`
- - start using `python zeronet.py`
 
+### Windows
+
+* [Install Python 2.7](https://www.python.org/ftp/python/2.7.9/python-2.7.9.msi)
+* [Install Python ZeroMQ](http://zeronet.io/files/windows/pyzmq-14.4.1.win32-py2.7.exe)
+* [Install Python Greenlet](http://zeronet.io/files/windows/greenlet-0.4.5.win32-py2.7.exe)
+* [Install Python Gevent](http://zeronet.io/files/windows/gevent-1.0.1.win32-py2.7.exe)
+* [Install Python MsgPack](http://zeronet.io/files/windows/msgpack-python-0.4.2.win32-py2.7.exe)
+* Start `start.py`
+
+### Linux
+
+#### Debian
+
+* `apt-get install python-dev python-pip` 
+* `pip install pyzmq gevent msgpack-python`
+* Start with `python zeronet.py`
+
+#### Without root access
+
+* `wget https://bootstrap.pypa.io/get-pip.py` 
+* `python get-pip.py --user pyzmq gevent msgpack-python`
+* Start with `python zeronet.py`
 
 ## Current limitations
- - No torrent-like, file splitting big file support
- - Just as anonymous as the bittorrent
- - File transactions not compressed or encrypted yet
- - No private sites
- - You must have an open port to publish new changes
- - Timeout problems on slow connections
+
+* No torrent-like, file splitting for big file support
+* No more anonymous than Bittorrent
+* File transactions are not compressed or encrypted yet
+* No private sites
+* You must have an open port to publish new changes
+* Timeout problems on slow connections
 
 
 ## How can I create a ZeroNet site?
+
 Shut down zeronet if you are running it already
-```
+
+```bash
 $ zeronet.py siteCreate
 ...
 - Site private key: 23DKQpzxhbVBrAtvLEc2uvk7DZweh4qL3fn3jpM3LgHDczMK2TtYUq
@@ -67,27 +85,35 @@ $ zeronet.py siteCreate
 $ zeronet.py
 ...
 ```
-Congratulations, you are done! Now anyone can access your site using http://localhost:43110/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
+
+Congratulations, you're finished! Now anyone can access your site using
+`http://localhost:43110/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2`
 
 Next steps: [ZeroNet Developer Documentation](https://github.com/HelloZeroNet/ZeroNet/wiki/ZeroNet-Developer-Documentation)
 
 
 ## How can I modify a ZeroNet site?
-- Modify files located in data/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2 directory. After you done:
-```
+
+* Modify files located in data/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2 directory.
+  After you're finished:
+
+```bash
 $ zeronet.py siteSign 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
 - Signing site: 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2...
 Private key (input hidden):
 ```
-- Enter your private key you got when created the site, then:
-```
+
+* Enter the private key you got when created the site, then:
+
+```bash
 $ zeronet.py sitePublish 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
 ...
 Site:13DNDk..bhC2 Publishing to 3/10 peers...
 Site:13DNDk..bhC2 Successfuly published to 3 peers
 - Serving files....
 ```
-- That's it! You successfuly signed and published your modifications.
+
+* That's it! You've successfully signed and published your modifications.
 
 
 ## If you want to help keep this project alive
@@ -97,5 +123,5 @@ Bitcoin: 1QDhxQ6PraUZa21ET5fYUCPgdrwBomnFgX
 
 #### Thank you!
 
-- More info, help, changelog, zeronet sites: http://www.reddit.com/r/zeronet/
-- Come, chat with us: [#zeronet @ FreeNode](https://kiwiirc.com/client/irc.freenode.net/zeronet) 
+* More info, help, changelog, zeronet sites: http://www.reddit.com/r/zeronet/
+* Come, chat with us: [#zeronet @ FreeNode](https://kiwiirc.com/client/irc.freenode.net/zeronet) 

src/Config.py

@@ -3,7 +3,7 @@
 
 class Config(object):
 	def __init__(self):
-		self.version = "0.2.1"
+		self.version = "0.2.3"
 		self.parser = self.createArguments()
 		argv = sys.argv[:] # Copy command line arguments
 		argv = self.parseConfig(argv) # Add arguments from config file
@@ -19,7 +19,7 @@ def __str__(self):
 	def createArguments(self):
 		# Platform specific
 		if sys.platform.startswith("win"):
-			upnpc = "tools\\upnpc\\upnpc-static.exe"
+			upnpc = "tools\\upnpc\\upnpc-shared.exe"
 			coffeescript = "type %s | tools\\coffee\\coffee.cmd"
 		else:
 			upnpc = None

src/File/FileServer.py

@@ -18,6 +18,7 @@ def __init__(self):
 		else:
 			self.port_opened = None # Is file server opened on router
 		self.sites = SiteManager.list()
+		self.running = True
 
 
 	# Handle request to fileserver
@@ -168,17 +169,26 @@ def start(self, check_sites = True):
 		if check_sites: # Open port, Update sites, Check files integrity
 			gevent.spawn(self.checkSites)
 
-		gevent.spawn(self.announceSites)
-		gevent.spawn(self.wakeupWatcher)
+		thread_announce_sites = gevent.spawn(self.announceSites)
+		thread_wakeup_watcher = gevent.spawn(self.wakeupWatcher)
 
-		while True:
+		while self.running:
 			try:
 				ret = {}
 				req = msgpack.unpackb(socket.recv())
 				self.handleRequest(req)
 			except Exception, err:
 				self.log.error(err)
-				self.socket.send(msgpack.packb({"error": "%s" % Debug.formatException(err)}, use_bin_type=True))
+				if self.running: self.socket.send(msgpack.packb({"error": "%s" % Debug.formatException(err)}, use_bin_type=True))
 				if config.debug: # Raise exception
 					import sys
-					sys.excepthook(*sys.exc_info())
+					sys.modules["src.main"].DebugHook.handleError() 
+		thread_wakeup_watcher.kill(exception=Debug.Notify("Stopping FileServer"))
+		thread_announce_sites.kill(exception=Debug.Notify("Stopping FileServer"))
+		self.log.debug("Stopped.")
+
+
+	def stop(self):
+		self.running = False
+		self.socket.close()
+

src/Site/Site.py

@@ -194,9 +194,6 @@ def publisher(self, inner_path, peers, published, limit):
 				self.log.info("[OK] %s: %s" % (peer.key, result["ok"]))
 			else:
 				self.log.info("[ERROR] %s: %s" % (peer.key, result))
-
-
-
 
 
 	# Update content.json on peers

src/Ui/UiServer.py

@@ -11,20 +11,25 @@
 # Skip websocket handler if not necessary
 class UiWSGIHandler(WSGIHandler):
 	def __init__(self, *args, **kwargs):
+		self.server = args[2]
 		super(UiWSGIHandler, self).__init__(*args, **kwargs)
-		self.ws_handler = WebSocketHandler(*args, **kwargs)
+		self.args = args
+		self.kwargs = kwargs
 
 
 	def run_application(self):
+		self.server.sockets[self.client_address] = self.socket
 		if "HTTP_UPGRADE" in self.environ: # Websocket request
-			self.ws_handler.__dict__ = self.__dict__ # Match class variables
-			self.ws_handler.run_application()
+			ws_handler = WebSocketHandler(*self.args, **self.kwargs)
+			ws_handler.__dict__ = self.__dict__ # Match class variables
+			ws_handler.run_application()
 		else: # Standard HTTP request
 			#print self.application.__class__.__name__
 			try:
 				return super(UiWSGIHandler, self).run_application()
 			except Exception, err:
 				logging.debug("UiWSGIHandler error: %s" % err)
+		del self.server.sockets[self.client_address]
 
 
 class UiServer:
@@ -89,4 +94,27 @@ def start(self):
 				browser = webbrowser.get(config.open_browser)
 			browser.open("http://%s:%s" % (config.ui_ip, config.ui_port), new=2) 
 
-		WSGIServer((self.ip, self.port), handler, handler_class=UiWSGIHandler, log=self.log).serve_forever()
+		self.server = WSGIServer((self.ip, self.port), handler, handler_class=UiWSGIHandler, log=self.log)
+		self.server.sockets = {}
+		self.server.serve_forever()
+		self.log.debug("Stopped.")
+
+	def stop(self):
+		# Close WS sockets
+		for client in self.server.clients.values():
+			client.ws.close()
+		# Close http sockets
+		sock_closed = 0
+		for sock in self.server.sockets.values():
+			try:
+				sock._sock.close()
+				sock.close()
+				sock_closed += 1
+			except Exception, err:
+				pass
+		self.log.debug("Socket closed: %s" % sock_closed)
+
+		self.server.socket.close()
+		self.server.stop()
+		time.sleep(1)
+

src/Ui/UiWebsocket.py

@@ -121,6 +121,8 @@ def handleRequest(self, data):
 			func = self.actionSiteSetLimit
 		elif cmd == "channelJoinAllsite" and "ADMIN" in permissions:
 			func = self.actionChannelJoinAllsite
+		elif cmd == "serverUpdate" and "ADMIN" in permissions:
+			func = self.actionServerUpdate
 		# Unknown command
 		else:
 			self.response(req["id"], "Unknown command: %s" % cmd)
@@ -361,3 +363,12 @@ def actionSiteSetLimit(self, to, size_limit):
 		self.site.saveSettings()
 		self.response(to, "Site size limit changed to %sMB" % size_limit)
 		self.site.download()
+
+
+	def actionServerUpdate(self, to):
+		import sys
+		self.cmd("updating")
+		sys.modules["src.main"].update_after_shutdown = True
+		sys.modules["src.main"].file_server.stop()
+		sys.modules["src.main"].ui_server.stop()
+

src/Ui/media/Wrapper.coffee

@@ -46,6 +46,9 @@ class Wrapper
 			@sendInner message # Pass to inner frame
 			if message.params.address == window.address # Current page
 				@setSiteInfo message.params
+		else if cmd == "updating" # Close connection
+			@ws.ws.close()
+			@ws.onCloseWebsocket(null, 4000)
 		else
 			@sendInner message # Pass message to inner frame
 
@@ -69,6 +72,10 @@ class Wrapper
 			@actionWrapperPrompt(message)
 		else if cmd == "wrapperSetViewport" # Set the viewport
 			@actionSetViewport(message)
+		else if cmd == "wrapperGetLocalStorage"
+			@actionGetLocalStorage(message)
+		else if cmd == "wrapperSetLocalStorage"
+			@actionSetLocalStorage(message)			
 		else # Send to websocket
 			if message.id < 1000000
 				@ws.send(message) # Pass message to websocket
@@ -126,6 +133,16 @@ class Wrapper
 			$('<meta name="viewport" id="viewport">').attr("content", @toHtmlSafe message.params).appendTo("head")
 
 
+	actionGetLocalStorage: (message) ->
+		data = localStorage.getItem "site.#{window.address}"
+		if data then data = JSON.parse(data)
+		@sendInner {"cmd": "response", "to": message.id, "result": data}
+
+
+	actionSetLocalStorage: (message) ->
+		back = localStorage.setItem "site.#{window.address}", JSON.stringify(message.params)
+
+
 	# EOF actions
 
 
@@ -152,7 +169,7 @@ class Wrapper
 		@wrapperWsInited = false
 		setTimeout (=> # Wait a bit, maybe its page closing
 			@sendInner {"cmd": "wrapperClosedWebsocket"} # Send to inner frame
-			if e.code == 1000 # Server error please reload page
+			if e and e.code == 1000 and e.wasClean == false # Server error please reload page
 				@ws_error = @notifications.add("connection", "error", "UiServer Websocket error, please reload the page.")
 			else if not @ws_error
 				@ws_error = @notifications.add("connection", "error", "Connection with <b>UiServer Websocket</b> was lost. Reconnecting...")