gaspersic/go-ethereum
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
10,511 Commits 29 Branches 198 Tags
7ae2a7bd84e7ee738874916dd07c9f2020c4fabb
Commit Graph

310 Commits

Author SHA1 Message Date
lash
7ae2a7bd84 swarm: Reinstate Pss Protocol add call through swarm service (#19117) 
* swarm: Reinstate Pss Protocol add call through swarm service

* swarm: Even less self

(cherry picked from commit d88c6ce6b0)
 2019-02-19 13:18:10 +01:00
Kiel barry
fd34bf594c contracts/*: golint updates for this or self warning 
(cherry picked from commit 53b823afc8)
 2019-02-19 13:18:02 +01:00
Ferenc Szabo
8857707606 p2p, swarm: fix node up races by granular locking (#18976) 
* swarm/network: DRY out repeated giga comment

I not necessarily agree with the way we wait for event propagation.
But I truly disagree with having duplicated giga comments.

* p2p/simulations: encapsulate Node.Up field so we avoid data races

The Node.Up field was accessed concurrently without "proper" locking.
There was a lock on Network and that was used sometimes to access
the  field. Other times the locking was missed and we had
a data race.

For example: https://github.com/ethereum/go-ethereum/pull/18464
The case above was solved, but there were still intermittent/hard to
reproduce races. So let's solve the issue permanently.

resolves: ethersphere/go-ethereum#1146

* p2p/simulations: fix unmarshal of simulations.Node

Making Node.Up field private in 13292ee897
broke TestHTTPNetwork and TestHTTPSnapshot. Because the default
UnmarshalJSON does not handle unexported fields.

Important: The fix is partial and not proper to my taste. But I cut
scope as I think the fix may require a change to the current
serialization format. New ticket:
https://github.com/ethersphere/go-ethereum/issues/1177

* p2p/simulations: Add a sanity test case for Node.Config UnmarshalJSON

* p2p/simulations: revert back to defer Unlock() pattern for Network

It's a good patten to call `defer Unlock()` right after `Lock()` so
(new) error cases won't miss to unlock. Let's get back to that pattern.

The patten was abandoned in 85a79b3ad3,
while fixing a data race. That data race does not exist anymore,
since the Node.Up field got hidden behind its own lock.

* p2p/simulations: consistent naming for test providers Node.UnmarshalJSON

* p2p/simulations: remove JSON annotation from private fields of Node

As unexported fields are not serialized.

* p2p/simulations: fix deadlock in Network.GetRandomDownNode()

Problem: GetRandomDownNode() locks -> getDownNodeIDs() ->
GetNodes() tries to lock -> deadlock

On Network type, unexported functions must assume that `net.lock`
is already acquired and should not call exported functions which
might try to lock again.

* p2p/simulations: ensure method conformity for Network

Connect* methods were moved to p2p/simulations.Network from
swarm/network/simulation. However these new methods did not follow
the pattern of Network methods, i.e., all exported method locks
the whole Network either for read or write.

* p2p/simulations: fix deadlock during network shutdown

`TestDiscoveryPersistenceSimulationSimAdapter` often got into deadlock.
The execution was stuck on two locks, i.e, `Kademlia.lock` and
`p2p/simulations.Network.lock`. Usually the test got stuck once in each
20 executions with high confidence.

`Kademlia` was stuck in `Kademlia.EachAddr()` and `Network` in
`Network.Stop()`.

Solution: in `Network.Stop()` `net.lock` must be released before
calling `node.Stop()` as stopping a node (somehow - I did not find
the exact code path) causes `Network.InitConn()` to be called from
`Kademlia.SuggestPeer()` and that blocks on `net.lock`.

Related ticket: https://github.com/ethersphere/go-ethereum/issues/1223

* swarm/state: simplify if statement in DBStore.Put()

* p2p/simulations: remove faulty godoc from private function

The comment started with the wrong method name.

The method is simple and self explanatory. Also, it's private.
=> Let's just remove the comment.

(cherry picked from commit 50b872bf05)
 2019-02-19 13:11:52 +01:00
gluk256
d6c1fcbe04 swarm/pss: refactoring (#19110) 
* swarm/pss: split pss and keystore

* swarm/pss: moved whisper to keystore

* swarm/pss: goimports fixed

(cherry picked from commit 12ca3b172a)
 2019-02-19 13:11:52 +01:00
Elad
79cac793c0 swarm/storage/netstore: add fetcher cancellation on shutdown (#19049) 
swarm/network/stream: remove netstore internal wg
swarm/network/stream: run individual tests with t.Run

(cherry picked from commit 3ee09ba035)
 2019-02-19 13:11:52 +01:00
holisticode
5de6b6b529 swarm/network: Saturation check for healthy networks (#19071) 
* swarm/network: new saturation for  implementation

* swarm/network: re-added saturation func in Kademlia as it is used elsewhere

* swarm/network: saturation with higher MinBinSize

* swarm/network: PeersPerBin with depth check

* swarm/network: edited tests to pass new saturated check

* swarm/network: minor fix saturated check

* swarm/network/simulations/discovery: fixed renamed RPC call

* swarm/network: renamed to isSaturated and returns bool

* swarm/network: early depth check

(cherry picked from commit 2af24724dd)
 2019-02-19 13:11:52 +01:00
Elad
3d2bedf8d0 swarm/storage: fix influxdb gc metrics report (#19102) 
(cherry picked from commit 5b8ae7885e)
 2019-02-19 13:11:52 +01:00
Janoš Guljaš
8ea3d8ad90 swarm: fix network/stream data races (#19051) 
* swarm/network/stream: newStreamerTester cleanup only if err is nil

* swarm/network/stream: raise newStreamerTester waitForPeers timeout

* swarm/network/stream: fix data races in GetPeerSubscriptions

* swarm/storage: prevent data race on LDBStore.batchesC

https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-461775049

* swarm/network/stream: fix TestGetSubscriptionsRPC data race

https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-461768477

* swarm/network/stream: correctly use Simulation.Run callback

https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-461783804

* swarm/network: protect addrCountC in Kademlia.AddrCountC function

https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-462273444

* p2p/simulations: fix a deadlock calling getRandomNode with lock

https://github.com/ethersphere/go-ethereum/issues/1198#issuecomment-462317407

* swarm/network/stream: terminate disconnect goruotines in tests

* swarm/network/stream: reduce memory consumption when testing data races

* swarm/network/stream: add watchDisconnections helper function

* swarm/network/stream: add concurrent counter for tests

* swarm/network/stream: rename race/norace test files and use const

* swarm/network/stream: remove watchSim and its panic

* swarm/network/stream: pass context in watchDisconnections

* swarm/network/stream: add concurrent safe bool for watchDisconnections

* swarm/storage: fix LDBStore.batchesC data race by not closing it

(cherry picked from commit 3fd6db2bf6)
 2019-02-19 13:11:52 +01:00
Elad
a0127019c3 swarm: fix uptime gauge update goroutine leak by introducing cleanup functions (#19040) 
(cherry picked from commit d596bea2d5)
 2019-02-19 13:11:51 +01:00
holisticode
7a333e4104 swarm/storage: fix HashExplore concurrency bug ethersphere#1211 (#19028) 
* swarm/storage: fix HashExplore concurrency bug ethersphere#1211

*  swarm/storage: lock as value not pointer

* swarm/storage: wait for  to complete

* swarm/storage: fix linter problems

* swarm/storage: append to nil slice

(cherry picked from commit 3d22a46c94)
 2019-02-19 13:11:51 +01:00
gluk256
799fe99537 swarm/pss: mutex lifecycle fixed (#19045) 
(cherry picked from commit b30109df3c)
 2019-02-19 13:11:51 +01:00
Rafael Matias
3b02b0ba4b swarm/docker: add global-store and split docker images (#19038) 
(cherry picked from commit 6cb7d52a29)
 2019-02-19 13:11:51 +01:00
Janoš Guljaš
85217b08bd cmd/swarm/global-store: global store cmd (#19014) 
(cherry picked from commit 33d0a0efa6)
 2019-02-19 13:11:51 +01:00
Ferenc Szabo
dcff622d43 swarm: CI race detector test adjustments (#19017) 
(cherry picked from commit 27e3f96819)
 2019-02-19 13:11:51 +01:00
Anton Evangelatov
a3db00f270 swarm/network: refactor simulation tests bootstrap (#18975) 
(cherry picked from commit 597597e8b2)
 2019-02-19 13:11:50 +01:00
holisticode
769e43e334 swarm: GetPeerSubscriptions RPC (#18972) 
(cherry picked from commit 43e1b7b124)
 2019-02-19 13:11:50 +01:00
gluk256
8d8ddea1a3 swarm/pss: transition to whisper v6 (#19023) 
(cherry picked from commit cde02e017e)
 2019-02-19 13:09:10 +01:00
lash
068725c5b0 swarm/network, swarm/storage: Preserve opentracing contexts (#19022) 
(cherry picked from commit 0c10d37606)
 2019-02-19 13:09:09 +01:00
Ferenc Szabo
710775f435 swarm/network: fix data race in fetcher_test.go (#18469) 
(cherry picked from commit 19bfcbf911)
 2019-02-19 13:09:09 +01:00
lash
0fd0108507 swarm/pss: Remove pss service leak in test (#18992) 
(cherry picked from commit 7c60d0a6a2)
 2019-02-19 13:06:14 +01:00
Ferenc Szabo
3c62cc6bba swarm/storage: fix test timeout with -race by increasing mget timeout 
(cherry picked from commit 1c3aa8d9b1)
 2019-02-19 13:06:14 +01:00
Janoš Guljaš
333b1bfb6c swarm/storage/localstore: new localstore package (#19015) 
(cherry picked from commit 4f3d22f06c)
 2019-02-19 13:06:14 +01:00
holisticode
d1ace4f344 swarm: Debug API and HasChunks() API endpoint (#18980) 
(cherry picked from commit 41597c2856)
 2019-02-19 13:06:13 +01:00
holisticode
1ecf2860cf cmd/swarm: hashes command (#19008) 
(cherry picked from commit 7f55b0cbd8)
 2019-02-19 12:57:53 +01:00
holisticode
034f65e9e8 swarm/storage: Get all chunk references for a given file (#19002) 
(cherry picked from commit 3eff652a7b)
 2019-02-19 12:57:53 +01:00
lash
607a1968e6 swarm/network: Remove extra random peer, connect test sanity, comments (#18964) 
(cherry picked from commit f9401ae011)
 2019-02-19 12:57:53 +01:00
Janoš Guljaš
3f54994db0 swarm: fix flaky delivery tests (#18971) 
(cherry picked from commit 592bf6a59c)
 2019-02-19 12:57:53 +01:00
Elad
2695aa9e0d p2p/testing, swarm: remove unused testing.T in protocol tester (#18500) 
(cherry picked from commit 2abeb35d54)
 2019-02-19 12:56:31 +01:00
gluk256
e247dcc141 swarm/version: commit version added (#18510) 
(cherry picked from commit ad13d2d407)
 2019-02-19 12:56:31 +01:00
Janoš Guljaš
b774d0a507 swarm: fix a data race on startTime (#18511) 
(cherry picked from commit fa34429a26)
 2019-02-19 12:56:30 +01:00
Anton Evangelatov
4976fcc91a swarm: bootnode-mode, new bootnodes and no p2p package discovery (#18498) 
(cherry picked from commit bbd120354a)
 2019-02-19 12:56:30 +01:00
Elad
afb65f6ace swarm/network: fix data race warning on TestBzzHandshakeLightNode (#18459) 
(cherry picked from commit 81e26d5a48)
 2019-02-19 12:55:18 +01:00
Viktor Trón
1f1c751b6e swarm/network: rewrite of peer suggestion engine, fix skipped tests (#18404) 
* swarm/network: fix skipped tests related to suggestPeer

* swarm/network: rename depth to radius

* swarm/network: uncomment assertHealth and improve comments

* swarm/network: remove commented code

* swarm/network: kademlia suggestPeer algo correction

* swarm/network: kademlia suggest peer

 * simplify suggest Peer code
 * improve peer suggestion algo
 * add comments
 * kademlia testing improvements
   * assertHealth -> checkHealth (test helper)
   * testSuggestPeer -> checkSuggestPeer (test helper)
   * remove testSuggestPeerBug and TestKademliaCase

* swarm/network: kademlia suggestPeer cleanup, improved comments

* swarm/network: minor comment, discovery test default arg

(cherry picked from commit bcb2594151)
 2019-02-19 12:55:07 +01:00
Elad
a3f31f51f3 cmd/swarm/swarm-snapshot: swarm snapshot generator (#18453) 
* cmd/swarm/swarm-snapshot: add binary to create network snapshots

* cmd/swarm/swarm-snapshot: refactor and extend tests

* p2p/simulations: remove unused triggerChecks func and fix linter

* internal/cmdtest: raise the timeout for killing TestCmd

* cmd/swarm/swarm-snapshot: add more comments and other minor adjustments

* cmd/swarm/swarm-snapshot: remove redundant check in createSnapshot

* cmd/swarm/swarm-snapshot: change comment wording

* p2p/simulations: revert Simulation.Run from master

https://github.com/ethersphere/go-ethereum/pull/1077/files#r247078904

* cmd/swarm/swarm-snapshot: address pr comments

* swarm/network/simulations/discovery: removed snapshot write to file

* cmd/swarm/swarm-snapshot, swarm/network/simulations: removed redundant connection event check, fixed lint error

(cherry picked from commit 34f11e752f)
 2019-02-19 12:54:56 +01:00
Janoš Guljaš
e63995b3f3 swarm/network: fix data race in TestNetworkID test (#18460) 
(cherry picked from commit 96c7c18b18)
 2019-02-19 12:54:10 +01:00
Janoš Guljaš
dd3e894747 swarm/storage: fix mockNetFetcher data races (#18462) 
fixes: ethersphere/go-ethereum#1117
(cherry picked from commit f728837ee6)
 2019-02-19 12:54:10 +01:00
Felix Lange
7fa3509e2e params, swarm/version: Geth 1.8.22-stable, Swarm 0.3.10-stable 2019-01-31 11:52:18 +01:00
Péter Szilágyi
9dc5d1a915 params, swarm: release Geth v1.8.21 and Swarm v0.3.9 2019-01-15 22:51:31 +02:00
gluk256
4aeeecfded swarm/pot: each() functions refactored (#18452) 2019-01-15 11:51:33 +01:00
gluk256
1636d9574b swarm/pot: pot.remove fixed (#18431) 
* swarm/pot: refactored pot.remove(), updated comments

* swarm/pot: comments updated
 2019-01-11 20:42:33 +01:00
holisticode
88168ff5c5 Stream subscriptions (#18355) 
* swarm/network: eachBin now starts at kaddepth for nn

* swarm/network: fix Kademlia.EachBin

* swarm/network: fix kademlia.EachBin

* swarm/network: correct EachBin implementation according to requirements

* swarm/network: less addresses simplified tests

* swarm: calc kad depth outside loop in EachBin test

* swarm/network: removed printResults

* swarm/network: cleanup imports

* swarm/network: remove kademlia.EachBin; fix RequestSubscriptions and add unit test

* swarm/network/stream: address PR comments

* swarm/network/stream: package-wide subscriptionFunc

* swarm/network/stream: refactor to kad.EachConn
 2019-01-11 15:08:09 +01:00
Ferenc Szabo
2eb838ed97 p2p/simulations: eliminate concept of pivot (#18426) 2019-01-11 10:23:45 +01:00
lash
7240f4d800 swarm/network: Rename minproxbinsize, add as member of simulation (#18408) 
* swarm/network: Rename minproxbinsize, add as member of simulation

* swarm/network: Deactivate WaitTillHealthy, unreliable pending suggestpeer
 2019-01-10 12:33:51 +01:00
Viktor Trón
6df3e4eeb0 swarm/network: remove isproxbin bool from kad.Each* iterfunc (#18239) 
* swarm/network, swarm/pss: remove isproxbin bool from kad.Each* iterfunc

* swarm/network: restore comment and unskip snapshot sync tests
 2019-01-10 03:36:19 +01:00
Janoš Guljaš
d70c4faf20 swarm: Fix T.Fatal inside a goroutine in tests (#18409) 
* swarm/storage: fix T.Fatal inside a goroutine

* swarm/network/simulation: fix T.Fatal inside a goroutine

* swarm/network/stream: fix T.Fatal inside a goroutine

* swarm/network/simulation: consistent failures in TestPeerEventsTimeout

* swarm/network/simulation: rename sendRunSignal to triggerSimulationRun
 2019-01-09 07:05:55 +01:00
holisticode
ae857e74bf swarm, p2p/protocols: Stream accounting (#18337) 
* swarm: completed 1st phase of swap accounting

* swarm, p2p/protocols: added stream pricing

* swarm/network/stream: gofmt simplify stream.go

* swarm: fixed review comments

* swarm: used snapshots for swap tests

* swarm: custom retrieve for swap (less cascaded requests at any one time)

* swarm: addressed PR comments

* swarm: log output formatting

* swarm: removed parallelism in swap tests

* swarm: swap tests simplification

* swarm: removed swap_test.go

* swarm/network/stream: added prefix space for comments

* swarm/network/stream: unit test for prices

* swarm/network/stream: don't hardcode price

* swarm/network/stream: fixed invalid price check
 2019-01-08 00:59:00 +01:00
Janoš Guljaš
56a3f6c03c swarm/storage/mock/test: fix T.Fatal inside a goroutine (#18399) 2019-01-07 14:32:01 +01:00
Janoš Guljaš
356c49fa7e swarm: Shed Index and Uint64Field additions (#18398) 2019-01-07 13:20:11 +01:00
Ferenc Szabo
fe03b76ffe A few minor code inspection fixes (#18393) 
* swarm/network: fix code inspection problems

- typos
- redundant import alias

* p2p/simulations: fix code inspection problems

- typos
- unused function parameters
- redundant import alias
- code style issue: snake case

* swarm/network: fix unused method parameters inspections
 2019-01-06 11:58:57 +01:00
Dave McGregor
33d233d3e1 vendor, crypto, swarm: switch over to upstream sha3 package 2019-01-04 09:26:07 +02:00