Output clg #229

xh3b4sd · 2016-09-25T13:06:49Z

abstract

This PR goes towards #172.

fixes

This PR fixes #193.

checklist

Make sure you read and understood the CONTRIBUTING.md.
Make sure you applied sufficient labels, milestone and assignee.
Make sure all TODOs are addressed.
Make sure there is sufficient documentation.
Make sure there are sufficient tests.
Make sure all commits are squashed before merging.
Make sure to delete the feature branch after merging.

xh3b4sd · 2016-09-25T13:09:34Z

context/context.go

-	SessionID string
+	Context     netcontext.Context
+	Expectation spec.Expectation
+	SessionID   string

 	// TODO we want to track the original input that was provided from the


The original input needs to be tracked to forward the correct network payload from the output CLG to the input CLG in case a provided expectation was not met.

Now we lookup the original input using the information ID tracked in the context and send it to the input CLG referenced by the CLG tree ID tracked by the context.

xh3b4sd · 2016-09-25T13:10:59Z

network/activate.go

@@ -28,6 +28,9 @@ func (n *network) payloadFromConnections(behaviorID string, queue []spec.Network
 	// storage. Here it is important to obtain the order of the behavior IDs
 	// stored as connections. They represent the input interface of the requested
 	// CLG.
+	//
+	// TODO there needs to be some sort of variation when executing existing CLG trees


We need to add variations to the selection process of network payloads being merged together. The requested CLG needs to be executed with different inputs from time to time, regardless known successful configurations.

The implementation changed. There is now spec.Activator to manage the activation process.

xh3b4sd · 2016-09-25T13:39:38Z

network/forward.go

+		// within several CLGs. That way the network creates its own connections
+		// based on learned experiences.
+		//
+		// TODO where are these connections coming from?


Lets add a comment.

There are no known connections when executing a new CLG tree. The first execution of a CLG tree records some sort of event log. Connections are only known beginning with the second execution of the same CLG tree. Then a replay of the recorded event log can be done.

The implementation changed. There is now spec.Forwarder to manage the forwarding process.

xh3b4sd · 2016-09-25T13:41:36Z

network/forward.go

+		// based on learned experiences.
+		//
+		// TODO where are these connections coming from?
+		// TODO if there are none, we need to find some randomly


We need to create new connections in case there are none.

The implementation changed. There is now spec.Forwarder to manage the forwarding process.

xh3b4sd · 2016-09-25T13:42:53Z

network/forward.go

+		//
+		// TODO where are these connections coming from?
+		// TODO if there are none, we need to find some randomly
+		// TODO there needs to be some sort of variation when executing existing CLG trees


Even if there are connections stored, we need to add variations to the selection process. That way known CLG trees can be improved.

The implementation changed. There is now spec.Forwarder to manage the forwarding process.

xh3b4sd · 2016-09-25T13:46:30Z

network/forward.go

+	}
+
+	// TODO fetch CLG tree ID using information ID
+	// TODO fetch input CLG ID (behaviorID) using CLG tree ID


We need to lookup the behavior ID of the input CLG of the current CLG tree. Using this behavior ID we can forward the network payload from the output CLG to the input CLG.

The implementation changed. There is now spec.Forwarder to manage the forwarding process.

xh3b4sd · 2016-09-25T13:53:43Z

network/listen.go

+				n.Log.WithTags(spec.Tags{C: nil, L: "E", O: n, V: 4}, "%#v", maskAny(err))
+				continue
+			}
+			// TODO write a new CLG tree ID and add it to context


We need to create a new CLG tree ID and connect it with the behavior ID of the input CLG.

codecov-io · 2016-09-25T19:21:51Z

Current coverage is 89.90% (diff: 62.18%)

Merging #229 into master will decrease coverage by 3.57%

Powered by Codecov. Last update ad4ee8e...db16f0a

xh3b4sd · 2016-10-06T22:49:31Z

context/context.go

@@ -15,8 +15,11 @@ import (
 type key string

 const (
-	behaviorIDKey    key = "behavior-id"
+	// TODO make the context marshalable


We need to make the context marshalable.

xh3b4sd · 2016-10-06T22:50:56Z

doc/diary/2016_09_28.md

+not clean. I now pursue an approach using a task system. I think this will give
+more controle over the communication processes within the neural network.
+
+TODO


We need to write the diary.

xh3b4sd · 2016-10-06T22:53:48Z

key/storage.go

@@ -4,6 +4,8 @@ import (
 	"fmt"
 )

+// TODO rename to NetworkKeyFormat, NewNetworkKey


We need to rename CLGKeyFormat to NetworkKeyFormat, etc..

xh3b4sd · 2016-10-06T22:59:11Z

network/forwarder/forwarder.go

+		return nil, maskAnyf(invalidBehaviourIDError, "must not be empty")
+	}
+	behaviourIDsKey := key.NewCLGKey("forward:configuration:behaviour-id:%s:behaviour-ids", behaviourID)
+	// TODO add closer instead of nil


Instead of using the closer we should check if the set methods make sense to be used here. Further there should be a method to fetch all elements of a set at once. The forwarder limits the number of elements stored per CLG anyway, so walking the elements of a set incrementally is not necessary.

There is now a spec.Storage.GetAllFromSet method.

xh3b4sd · 2016-10-06T22:59:42Z

network/forwarder/forwarder.go

+	return newNetworkPayloads, nil
+}
+
+// TODO this should probably be moved to the output CLG


We should move this probably to the output CLG.

xh3b4sd · 2016-10-06T23:00:13Z

spec/forwarder.go

+	//     NewNetworkPayloads
+	//
+	// TODO
+	Forward(CLG CLG, networkPayload NetworkPayload) error


We need more documentation here.

xh3b4sd · 2016-10-06T23:00:31Z

spec/network.go

+	// TODO
+	EventListener(canceler <-chan struct{}) error
+
+	// TODO


Write more docs.

xh3b4sd · 2016-10-06T23:00:50Z

storage/memory/storage.go

@@ -203,6 +202,26 @@ func (s *storage) GetStringMap(key string) (map[string]string, error) {
 	return nil, notFoundError
 }

+// TODO
+func (s *storage) PopFromList(key string) (string, error) {


xh3b4sd · 2016-10-06T23:01:06Z

storage/redis/storage.go

@@ -244,6 +244,54 @@ func (s *storage) GetStringMap(key string) (map[string]string, error) {
 	return result, nil
 }

+// TODO
+func (s *storage) PopFromList(key string) (string, error) {


… CLG

xh3b4sd · 2016-10-08T22:25:45Z

The PR is huge again. We stop here. The only additional thing left is now writing a diary entry. The next step is to add the event tracker to create more connections and make the forwarder actually work.

xh3b4sd · 2016-10-09T23:31:28Z

LGTM

xh3b4sd added 2 commits September 25, 2016 00:32

added output CLG

ac92ff5

golint

231041f

xh3b4sd self-assigned this Sep 25, 2016

xh3b4sd commented Sep 25, 2016

View reviewed changes

compiling

0287748

xh3b4sd added 24 commits September 25, 2016 21:30

Merge branch 'master' into output-clg

cd22a11

register output CLG

0f146ce

made context configuration more clean and fixed Clone

2f0204d

improved input CLG communication

3ca1805

improved context getter interface

7f90d72

added context to network interface

44f28b7

compiling

012f4a8

added new diary entry

13752b4

WIP pseudo code for task system structure

661cf0b

WIP pseudo code for task system structure

278d6e9

WIP pseudo code for task system structure

aa27629

WIP pseudo code for task system structure

b4387d6

WIP pseudo code for task system structure

45167ea

added spec.Activator to manage the activation process of requested CLGs

72fec9a

comments

4f8d5a4

gocyclo

8ae3c7a

collect all possible activation combinations and choose one random

9851a29

added spec.RandomFactory.CreateMax to create a single random number

84694cc

WIP pseudo code for forwarder

138090e

added spec.Forwarder to manage the forwarding process of requested CLGs

5a2e963

updated CLGs

7d4fa70

fixed factory after renaming

23ae6cd

added worker-pool as new pinned dependency

2452ad9

added json Marshaler and Unmarshaler to spec.NetworkPayload

31af760

xh3b4sd added 2 commits October 6, 2016 22:43

compiling

82657bc

injected configured gateway collection

5ee6838

xh3b4sd commented Oct 6, 2016

View reviewed changes

xh3b4sd added 11 commits October 6, 2016 23:32

comments

af54ae5

made spec.Context marshalable

044e960

comments

8197b1c

improved memory storage implementations

2607013

fixed CLG tests

ffc941c

further improved memory storage implementation

0d0b841

renamed key method

78a0a79

comments

2c9c8b1

moved re-queueing of network payloads for the input CLG to the output…

4a7b776

… CLG

comments

91b7e03

comments

a84ebfc

xh3b4sd added 12 commits October 8, 2016 22:41

CLG coverage

fafef02

CLG coverage

20ab6a0

typo

86df4e5

storage coverage

78bc8a5

storage coverage

b9f7418

wrote diary entry

e9f0194

increased gocyclo threshold

85465b0

WIP

08ed3dc

coverage

0116ac7

gocyclo

d5047e2

compiling

831d118

coverage

db16f0a

xh3b4sd merged commit 6b5c0bc into master Oct 9, 2016

xh3b4sd deleted the output-clg branch October 9, 2016 23:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Output clg #229

Output clg #229

xh3b4sd commented Sep 25, 2016 •

edited

Loading

xh3b4sd Sep 25, 2016

xh3b4sd Oct 6, 2016 •

edited

Loading

xh3b4sd Sep 25, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Sep 25, 2016 •

edited

Loading

xh3b4sd Oct 6, 2016 •

edited

Loading

xh3b4sd Sep 25, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Sep 25, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Sep 25, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Sep 25, 2016

xh3b4sd Oct 6, 2016

codecov-io commented Sep 25, 2016 •

edited

Loading

xh3b4sd Oct 6, 2016

xh3b4sd Oct 7, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 8, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 8, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 8, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 8, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 8, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 8, 2016

xh3b4sd Oct 6, 2016

xh3b4sd Oct 8, 2016

xh3b4sd commented Oct 8, 2016

xh3b4sd commented Oct 9, 2016

Output clg #229

Output clg #229

Conversation

xh3b4sd commented Sep 25, 2016 • edited Loading

abstract

fixes

checklist

Choose a reason for hiding this comment

xh3b4sd Oct 6, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xh3b4sd Sep 25, 2016 • edited Loading

Choose a reason for hiding this comment

xh3b4sd Oct 6, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Sep 25, 2016 • edited Loading

Current coverage is 89.90% (diff: 62.18%)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xh3b4sd commented Oct 8, 2016

xh3b4sd commented Oct 9, 2016

xh3b4sd commented Sep 25, 2016 •

edited

Loading

xh3b4sd Oct 6, 2016 •

edited

Loading

xh3b4sd Sep 25, 2016 •

edited

Loading

xh3b4sd Oct 6, 2016 •

edited

Loading

codecov-io commented Sep 25, 2016 •

edited

Loading