gaspersic/go-ethereum
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Move swarm content to legacy

Browse Source
This commit is contained in:
Adam Schmideg
2019-04-05 13:00:02 +02:00
parent 48b2c1fa57
commit 6e313f0a6d
10 changed files with 3 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/_legacy/IPFS--SWARM.md Normal file
View File

@ -0,0 +1 @@
see here https://github.com/ethersphere/go-ethereum/wiki/IPFS--SWARM

80
docs/_legacy/Swarm---TODO.md Normal file
View File

@ -0,0 +1,80 @@
---
title: Swarm TODO
---
# Sprint plan
# scope
- forwarding only (no recursive lookup and no connecting to new nodes, only working with active peers)
## TODO
- integrate new p2p
- write unit tests for protocol and netstore (without protocol)
- rework protocol errors using errs after PR merged
- integrate new p2p or develop branch after p2p merge
- integrate cademlia into hive / peer pool with new p2p
- work out timeouts and timeout encoding
- cli tools
- url bar and proxy
## CLI
- hooking into DPA local API
- running as a daemon accepting request via socket?
### -
## Encryption
- encryption gateway to incentivise encryption of public content
- xor encryption with random chunks
- in-memory encryption keys
- originator encryption for private content
## APIs
- DAPP API - js integration (Fabian, Alex)
- mist dapp storage scheme, url->hash mapping (Fabian, Alex) [URL scheme](URL-Scheme)
# Discuss alternatives
I suggest we each pick 2/3 and read up on their project status, features, useability, objectives, etc
- Is it even worth it to reinvent/reimplement the wheel?
- what features do we want now and in future
- roadmap
# Brainstorming
- storage economy, incentivisation, examples:
-- content owner pays recurring ether fee for storage.
-- scheme to reward content owner each time content is accessed. i.e accessing content would requires fee. this would reward popular content. should be optional though.
- dht - chain interaction
- proof of custody https://docs.google.com/document/d/1F81ulKEZFPIGNEVRsx0H1gl2YRtf0mUMsX011BzSjnY/edit
- proof of resources http://systemdocs.maidsafe.net/content/system_components/proof_of_resources.html
- nonoutsourceable proofs of storage as mining criteria
- proof of storage capacity directly rewarded by contract
- streaming, hash chains
- routing and learning graph traversal
- minimising hops
- forwarding strategies, optimising dispersion of requests
- lifetime of requests, renewals (repeated retrieval requests), expiry, reposting (repeated storage request)
- redundancy - store same data in multiple nodes (e.g 4x)
- the more accessed a content is, the more available it should be, should increase performance for popular content.
# Simulations
- full table homogeneous nodes network size vs density vs table size expected row-sizes
- forwarding strategy vs latency vs traffic
- stable table, dropout rate vs routing optimisation by precalculating subtables for all peers. expected distance change (proximity delta) per hop
## Swarm
How far does the analogy go?
swarm of bees | a decentralised network of peers
-------|------------
living in a hive | form a distributed preimage archive
where they | where they
gather pollen | gather data chunks which they
to produce honey | transform into a longer data stream (document)
they consume and store | they serve and store
buzzing bzz | using bzz as their communications protocol

69
docs/_legacy/Swarm---distributed-preimage-archive.md Normal file
View File

@ -0,0 +1,69 @@
---
title: Swarm - distributed preimage archive
---
# Resources
## Swarm, the name
- https://www.facebook.com/swarmcorp, http://swarm.fund/
- https://bitcointalk.org/index.php?topic=650143.0
- https://bitcoinmagazine.com/17956/swarm-1-rick-falkvinges-swarmops-project/
- http://www.amazon.co.uk/Swarmwise-Tactical-Manual-Changing-World/dp/1463533152/
## Docs & specs
- [Swarm TODO](Swarm---TODO)
- Dani & Viktor on public wiki: https://github.com/ethereum/wiki/wiki/Distributed-Preimage-Archive
- Dani on swarm hash: https://github.com/ethereum/wiki/wiki/Swarm-Hash
- Dani on incentive system: https://github.com/ethersphere/swarm/blob/master/doc/incentives.md
- [The swarm smart contract](Swarm-Contract)
- gav on url-hint https://github.com/ethereum/wiki/wiki/URL-Hint-Protocol
- Gav on public wiki: https://github.com/ethereum/cpp-ethereum/wiki/Swarm
- network (DEVp2p)
- [Peer-to-Peer](Peer-to-Peer)
- on kademlia: https://github.com/ethereum/wiki/wiki/Cademlia-Peer-Selection
## Talks
- https://twitter.com/ethereumproject/status/538030376858693633
- Dr. Daniel Nagy: Ethereum ÐΞVcon-0: Keeping the Public Record Safe and Accessible - https://www.youtube.com/watch?v=QzYZQ03ON2o&list=PLJqWcTqh_zKEjpSej3ddtDOKPRGl_7MhS&index=7&spfreload=10
## Forum
- empty as of 01/2015: https://forum.ethereum.org/categories/swarm
-
## Mentions, discussions
- http://www.reddit.com/r/ethereum/comments/2d4uyw/swarm_and_whisper/
- http://www.reddit.com/r/ethereum/comments/2ityfz/ethereum_swarm/
- https://www.maidsafe.org/t/ethereums-swarm-p2p-storage-and-whisper-p2p-messaging/1528
- Vitalik's blogpost of 08/2014 - https://blog.ethereum.org/2014/08/16/secret-sharing-erasure-coding-guide-aspiring-dropbox-decentralizer/
- Vitalik: 'Swarm is out-of-scope': https://www.reddit.com/r/ethereum/comments/2phvml/constructive_criticism_of_ethereum_project_not/cmwtfqq
- Vitalik on eth components, swarm at 4:00 http://www.naation.com/2015/02/02/ethereum-explained-with-vitalik-buterin-inventor-and-leader-of-the-ethereum-project/5764/
- https://www.youtube.com/watch?v=zgkmQ-jQJHk&feature=youtu.be
## Media
- https://twitter.com/jeffehh/status/565927366271467521
- https://twitter.com/avsa/status/566255260713627648
- https://twitter.com/zeligf/status/566042020909973504
- https://www.reddit.com/r/ethereum/comments/2wryru/eli5_how_is_ethereum_supposed_to_be_a_dropbox
- https://forum.ethereum.org/discussion/comment/7593/#Comment_7593
## Code
- bzz PR: https://github.com/ethereum/go-ethereum/pull/255,
- repo https://github.com/ethersphere/go-ethereum/tree/bzz/
- ethereum p2p: https://github.com/ethereum/go-ethereum/p2p
- peer selection, peer pool: https://github.com/ethereum/go-ethereum/pull/253
- p2p cademlia branch (discontinued): https://github.com/ethersphere/go-ethereum/tree/kademlia
- Felix's node discovery code: https://github.com/ethereum/go-ethereum/tree/develop/p2p/discover
# Alternatives
- storj - http://storj.io/
- maidsafe - http://maidsafe.net/
- ipfs - http://ipfs.io/, https://www.youtube.com/watch?v=Fa4pckodM9g, http://static.benet.ai/t/ipfs.pdf, https://github.com/jbenet/go-ipfs, https://www.youtube.com/watch?v=8CMxDNuuAiQ, https://www.reddit.com/r/ethereum/comments/2wot2i/ipfs_alpha_demo/
- filecoin - http://filecoin.io/
- permacoin - https://www.cs.umd.edu/~elaine/docs/permacoin.pdf, https://bitcointalk.org/index.php?topic=640410.0, http://blog.dshr.org/2014/06/permacoin.html
- siacoin - http://www.siacoin.com/
- riak - http://basho.com/riak/
- BitTorrent http://www.bittorrent.com/ maelstrom http://blog.bittorrent.com/2014/12/10/project-maelstrom-the-internet-we-build-next/
- Tahoe-LAFS https://www.tahoe-lafs.org/trac/tahoe-lafs
- retroshare http://retroshare.sourceforge.net/

166
docs/_legacy/Swarm-Channels,-Namereg-resolution-draft.md Normal file
View File

@ -0,0 +1,166 @@
---
title: Swarm channels, namereg-resolution draft
---
# Channels and streams
a *swarm chain* is an ordered list of content that are linked as a forkless chain.
.
This is simply modeled as linked manifests.
a *channel* is a sequence of manifests (_S_) and a relative path _P_ with a starting manifest _M_ and a streamsize _n_ (can be infinite). A channel is well-formed or regular if in every manifest in the stream _P_ resolves to a consistent mime type _T_ . For instance , if _T_ is `application/bzz-manifest+json`, we say the channel is a _manifest channel_, if the mime-type is `mpeg`, its a video channel.
A *primary channel* is a channel that actually respect chronological order of creation.
A *live channel* is a primary channel that keeps updating (adding episodes to the end of the chain) can have a (semi)-persistent mime-type for path _P_
A *blockstream channel* is a primary channel provable linked in time via hashes.
A *signed channel* is a primary channel provably linked by signatures (sequence position index signed by the publisher)
*Trackers* are a manifest channel which tracks updates to a primary channel and provides forward linking .
## Example channels:
- name histories, e.g updates of a domain, temporal snapshots of content
- blockchain: blockchain is a special case of blockstream
- git graph, versioning
- modeling a source of information: provable communication with hash chain, not allowed to fork, numbered.
#### content trackers
reverse index of a stream
- contains `next` links to following state
- published after next state
- publish provable quality metrics:
- age: starting date of tracker vs date of orig content
- neg exp forgetting(track date vs primary date of next episode) ~ alertness, puncuality (tracker
- git version control
every named host defines a timeline,
- create a manifest stream tracking a site
## Ways to link manifests
#### examples
``` json
{ "entries":
[
{
"host": "fefe.eth",
"number": 9067,
"previous": "ffca34987",
"next": "aefbc4569ab",
"this": "90daefaaabbc",
}
],
"auth": "3628aeefbc7689523aebc2489",
}
```
# Name resolution
The host part in a bzz webaddress should be resolved with our version of DNS, ie. using both `NameReg` (name registration contract on ethereum) and a simple mutable storage in swarm.
## signed version store
The point of channels (Swarm---Channels
) is to have a total order over a set of manifests.
The typical usecase is that it should be enough to know the name of a site or document to always see the latest version of a software or get the current episode of your favourite series or the consensus state of a blockchain. It should also be possible to deterministically derive the key to future content...
One possibility is to modify the NameReg entry in the blockchain to point to a swarm hash. Recording each change on the blockchain results in an implicit live channel linking. This scheme is simple inasmuch as it puts authentication completely on the chain. However, it is expensive and not viable given the number of publishers and typical rate of update.
Alternatively, swarm provides the protocol for associating a channel and a position with a swarm hash. The versioning can be authenticated since a message containing host name, sequence position index and swarm hash is signed by the public key registered in ethereum NameReg for the host.
Publishers, fans or paid bookkeepers track updates of a channel and wrap accumulated messages into a tracker manifest.
Most probably publishers would broadcast updates of a channel manifest in the first place.
This special key-value store can be implemented as a mutable store: the value with a higher index will simply override the previous one.
There can be various standards to derive lookup key deterministically
the simplest one is `HASH(host:version)` for a specific version and `HASH(host)` for the latest version.
The content has the following structure:
```
sign[<host>, <version>, <timestamp>, <hash>]
```
Retrieve request for a signed version is the same as a request for a hash.
[RetrieveMsg, HASH(host:version), id, timeout, MUTABLE]
[RetrieveMsg, HASH(host:0), id, timeout, MUTABLE]
Store request for a signed version is the same as for a hash:
[StoreMsg, key, id, MUTABLE, Sign[host, version, time.Unix(), hash]]
## Format
It is up to debate how we distinguish names to be resolved.
An early idea was to use a top level domain, such as `.eth` (<source> == `<host>.eth`)
this might limit the possibilities
Another idea was to have it as or part of the protocol: `eth://my-website.home` or `eth+bzz://my-website.home`. This are semantically incorrect, however.
Third, put an _eth_ inside the host somehow.
Ad-hoc constructs like `bzz://eth:my-website.home` will be rejected by host pattern matchers.
Abusing subdomains `bzz://eth.my-website.home` would cause ambiguity and potential collision.
Abusing auth `user:pass@my-website.home` would disable basic auth.
A suggestion that most aligns with the *signed versioning* and very simple is that we look up everything that is not a 32 byte hash format for a public key. The version of the site is looked up using the port part of the host. A specific version is given after the `:`.
The generic pattern then:
```
(<version_chain>.)<host>(:<number>)(/<path>)
```
### example 0
```
bzz://breaking.bad.tv/s4/e2/video
```
- _breaking.bad.tv_ is looked up in NameReg to yield public key _P_
- _breaking.bad.tv_ is looked up in the immutable store to yield a message `[_breaking.bad.tv_,0,3,aebf45fbf6ae6aaaafedcbcb467]`
- `aebf45fbf6ae6aaaafedcbcb467` is looked up in swarm to yield the manifest
- manifest entry for path `/s4/e2/video` results in the actual document's root key
### example 1
```
bzz://breaking.bad.tv/video
```
resolves the following way:
- _breaking.bad.tv_ is looked up in NameReg to yield public key _P_
- _breaking.bad.tv_ is looked up in the immutable store to yield - by `H(cookie)` - a message `[_breaking.bad.tv_,s5:e12,3,aebf45fbf6ae6aaaafedcbcb467]` signed by `P`
- `aebf45fbf6ae6aaaafedcbcb467` is looked up in swarm to yield the manifest
- manifest entry for path `video` results in the actual document's root key
### example 2
```
bzz://current.breaking.bad.tv:s4:e10/video
```
- _breaking.bad.tv_ is looked up in NameReg to yield public key _P_
- current.breaking.bad.tv:s4:e10 is looked up in the immutable store to yield a message `[current.breaking.bad.tv,s4:e10,3,45fbf6ae6aaaafedcbcb467ccc]`
- `45fbf6ae6aaaafedcbcb467ccc` is looked up in swarm to yield the manifest
- manifest entry for path `video` results in the actual document's root key
### example 3
```
bzz://breaking.bad.tv/playlist
```
- same as ex 2...
- manifest entry for path `playlist` results in a playlist manifest
### example 4
```
bzz://stable.ethereum.org:8.1/download/go/mac-os
```
- _stable.ethereum.org_ is looked up in NameReg to yield public key _P_
- stable.ethereum.org:s4:e10 is looked up in the immutable store to yield a message `[stable.ethereum.org,s4:e10,3,45fbf6ae6aaaafedcbcb467ccc]`
- `45fbf6ae6aaaafedcbcb467ccc` is looked up in swarm to yield the manifest
- manifest entry for path `video` results in the actual document's root key
Generalised content streaming, subscriptions.

40
docs/_legacy/Swarm-Contract.md Normal file
View File

@ -0,0 +1,40 @@
---
title: Swarm contract
---
This one contract regulates the incentive structure of Swarm.
The corresponding solidity code can be browsed [here](https://github.com/ethersphere/go-ethereum/blob/bzz/bzz/bzzcontract/swarm.sol).
# Methods
## Sign up as a node
Pay a deposit in Ether and register public key. Comes with an accessor for checking that a node is signed up.
## Demand penalty for loss of chunk
Present a signed receipt by a signed up node and a deposit covering the upload of a chunk. After a given deadline, the signer node's deposit is taken and the presenting node's deposit refunded, unless the chunk is presented. Comes with an accessor for checking that a given chunk has been reported lost, so that holders of receipts by other swarm nodes can punish them as well for losing the chunk, which, in turn, incentivizes whoever holds the chunk to present it.
## Present chunk to avoid penalty
No penalty is paid for lost chunks, if chunk is presented within the deadline. The cost of uploading the chunk is compensated exactly from the demand's deposit, with the remainder refunded. Comes with an accessor for checking that a given node is liable for penalty, so the node is notified to present the chunk in a timely fashion.
# Price considerations
For the price of accepting a chunk for storing, see [Incentives](https://github.com/ethersphere/swarm/blob/master/doc/incentives.md)
This price should be proportional to the sign-up deposit of the swarm node.
The deposit for compensating the swarm node for uploading the chunk into the block chain should be substantially higher (e.g. a small integer multiple) of the corresponding upload measured with the gas price used to upload the demand to prevent DoS attacks.
# Termination
Users of Swarm should be able to count on the loss of deposit as a disincentive, so it should not be refunded before the term of Swarm membership expires. If penalites were paid out as compensation to holders of receipts of lost chunks, it would provide an avenue of early exit for a Swarm member by "losing" chunks deposited by colluding users. Since users of Swarm are interested in their information being reliably stored, their primary incentive for keeping the receipts is to keep the Swarm motivated, not the potential compensation.
# Receipt circulation
End-users of Swarm keeping important information in it are obviously interested in keeping as many receipts of it as possible available for "litigation". The storage space required for storing a receipt is a sizable fraction of that used for storing the information itself, so end users can reduce their storage requirement further by storing the receipts in Swarm as well. Doing this recursively would result in end users only having to store a single receipt, yet being
able to penalize quite a few Swarm nodes, in case only a small part of their stored information
is lost.
Swarm nodes that use the rest of Swarm as a backup may want to propagate the receipts in the opposite direction of storage requests, so that the cost of storing receipts is eventually paid by the end user either in the form of allocated storage space or as a direct payment to Swarm.

60
docs/_legacy/swarm---POC-series.md Normal file
View File

@ -0,0 +1,60 @@
---
title: Swarm POC series
---
now project features and POC milestones are managed under
https://github.com/ethereum/go-ethereum/projects/6
## POC 0.1: plan bee
* [x] underground dev testnet launched on BZZ day (3.22 is l33t for BZZ)
* [x] create ansible and docker for node deployment
* [x] a few public gateways
## POC 0.2 sworm
* [x] rebase on geth 1.5 edge
* [x] small toynet deployed, see [network monitor](http://146.185.130.117/)
* [x] support for separate url schemes for dns enabled, immutable and raw manifest - feature [#2279](https://github.com/ethereum/go-ethereum/issues/2279)
* [x] Ethereum name service integration
* [x] scripted network tests, cluster control framework
* [x] algorithmic improvements on chunker split/join
* [x] algorithmic improvements on upload/download
* [x] algorithmic improvements in kademlia and hive peers manager
* [x] calibrating kademlia connectivity parameters to toynet scale
* [x] orange paper series released hosted on [swarm toynet](http://swarm-gateways.net/bzz:/swarm#the-thsphr-orange-papers)
* [x] minimalistic [swarm homepage](http://swarm-gateways.net/bzz:/swarm)
* [x] adapt to felix's rpc-client-as-eth-backend scheme to run swarm as a separate daemon
* [x] merge into main repo develop branch
## POC 0.3
Priorities are not finalised, this is just a tentative plan.
The following features are prioritised for POC 3 (subject to change)
* [ ] docker on azure: complete test cluster deployment and remote node control framework
* [ ] comprehensive suite of typical network scenarios
* [ ] syncer rewrite - syncdb refactor to storage
* [ ] mist integration
* [ ] storage monitoring and parameter setting API for Mist swarm dashboard
* [ ] bzz protocol should implement info for reporting - feature [#2042](https://github.com/ethereum/go-ethereum/issues/2042)
* [x] improved peer propagation [#2044](https://github.com/ethereum/go-ethereum/issues/2044)
* [x] abstract network simulation framework
* [x] protocol stack abstraction, pluggable subprotocol components - (swap, hive/protocol, syncer, peers) for pss
* [ ] new p2p API integration - feature [#2060](https://github.com/ethereum/go-ethereum/issues/2060)
* [ ] pss - unicast
* [ ] swap rewrite
* [ ] obfuscation for plausible deniability
## POC 0.4
* [ ] enhanced network monitoring, structured logging and stats aggregation
* [ ] unicast/multicast messaging over swarm - pss
* [ ] swear and swindle http://swarm-gateways.net/bzz:/swarm/ethersphere/orange-papers/1/sw%5E3.pdf
* [ ] smash/crash proof of custody http://swarm-gateways.net/bzz:/swarm/ethersphere/orange-papers/2/smash.pdf
* [ ] swarm DB support phase 0 - compact manifest trie and proof requests
* [ ] [SWarm On-demand Retrieval Daemon](https://gist.github.com/zelig/aa6eb43615e12d834d9f) - feature [#2049](https://github.com/ethereum/go-ethereum/issues/2049) = sword. ethereum state, contract storage, receipts, blocks on swarm
* [x] ~implement (a reviewed version of) [EIP-26](https://github.com/ethereum/EIPs/issues/26)~ obsoleted by ENS and the vickrey auction
* [ ] swarm namereg/natspec rewrite - enhancement [#2048](https://github.com/ethereum/go-ethereum/issues/2048)
* [x] ~solidity [contractInfo standard](https://github.com/ethereum/solidity/pull/645) and contract source verification support~ https://www.reddit.com/r/ethereum/comments/5d5lyd/first_contract_to_contain_swarm_hash_to_its/
## POC 0.5

1
docs/_legacy/swarm-dev-progress.md Normal file
View File

@ -0,0 +1 @@
See [Swarm Roadmap](https://github.com/orgs/ethersphere/projects/5)

1
docs/_legacy/swarm-roadmap.md Normal file
View File

@ -0,0 +1 @@
See [Swarm Roadmap](https://github.com/orgs/ethersphere/projects/5)