Commit Graph

555 Commits

Author SHA1 Message Date
NIC619
e1592997a8
Add type hints to routing folder 2019-07-30 16:24:52 +08:00
NIC619
f2de986c74
Add type hints to peer folder 2019-07-30 16:24:51 +08:00
NIC619
b695b0e1ec
Add type hint to host folder 2019-07-30 16:24:34 +08:00
NIC619
b2f496d081
Fix type hints except pb msg in pubsub folder 2019-07-30 16:24:34 +08:00
NIC619
a0aa105867
Add type hint to pubsub notifee/interface 2019-07-30 16:23:15 +08:00
NIC619
3549f2ff8b
Add type hints to mcache.py 2019-07-30 16:20:48 +08:00
NIC619
63014eeaae
Add type hints to floodsub.py 2019-07-30 16:20:47 +08:00
NIC619
b920955db6
Add type hints to gossipsub.py 2019-07-30 15:39:50 +08:00
NIC619
8eb6a230ff
Fix and add type hints to pubsub.py 2019-07-30 15:32:58 +08:00
mhchia
f02d38c0ee
Reflect PR feedback
* Rename `src` to `msg_forwarder` in pubsub/floodsub/gossipsub
* Rename Variables
* Sort imports
* Clean up
2019-07-29 12:09:35 +08:00
mhchia
74d831d4e2
Reflect PR feedback 2019-07-28 18:06:38 +08:00
Kevin Mai-Husan Chia
70c5c84f32
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:09:01 +08:00
Kevin Mai-Husan Chia
a1e20caebe
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:07:11 +08:00
Kevin Mai-Husan Chia
ffb3920468
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:06:03 +08:00
Kevin Mai-Husan Chia
c252c62009
Update libp2p/pubsub/pubsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:05:29 +08:00
mhchia
766d8ba1e1
A little bit clean up 2019-07-27 12:06:36 +08:00
mhchia
c028aef2de
Fix all tests
- Dedup `perform_test_from_obj` and the test cases used in both
`test_floodsub` and `test_gossipsub_backward_compatibility.py`.
Therefore, they are put in the standalone file
`tests/pubsub/floodsub_integration_test_settings.py`. The functions
and testcases are imported from there then.
- IMO still need a refactor on the tests. There are still some duplicate
code.
2019-07-27 11:49:03 +08:00
mhchia
65aedcb25a
Fix several tests 2019-07-26 18:35:25 +08:00
mhchia
035d08b8bd
Fix test_floodsub.py 2019-07-25 23:11:27 +08:00
mhchia
dadcf8138e
Fix the tests according to pubsub.Publish
And refactored a bit.
2019-07-25 16:58:00 +08:00
mhchia
cae4f34034
Refactor floodsub.publish
Passed the first test of floodsub
2019-07-25 14:08:16 +08:00
mhchia
93cf5a2c32
A roughly skeleton of floodsub.publish
Still need to ensure when to deliver to ourselves
2019-07-24 22:33:32 +08:00
mhchia
3f52b0dc0a
Remove leftover imports 2019-07-24 21:57:46 +08:00
mhchia
b528c211b9
Temp modified publish 2019-07-24 21:55:57 +08:00
mhchia
218bdb42c4
Add basic functionalities of publish 2019-07-24 21:55:04 +08:00
mhchia
8f9e5a28ff
Temp 2019-07-24 21:54:36 +08:00
Kevin Mai-Husan Chia
f329c5a627
Merge pull request #188 from mhchia/feature/add-typing-for-pubusb
Add tox and mypy
2019-07-24 21:45:19 +08:00
mhchia
d3a948be47
Fix error: Change params floodsub.publish back 2019-07-24 16:24:14 +08:00
mhchia
d6c19e71a6
Add typing and notes in pubsub 2019-07-24 14:54:30 +08:00
NIC619
99252e49f8
Prevent re-adding peers to mesh 2019-07-23 23:00:43 +08:00
NIC619
e52bfe3a51
Fix:
Respond GRAFT with PRUNE if not subscribed to the topic
2019-07-23 22:52:24 +08:00
NIC619
afc853a776
Apply PR feedback 2019-07-22 23:22:07 +08:00
NIC619
67f9edb77d
Remove fanout topic after joining the topic 2019-07-22 19:28:07 +08:00
NIC619
4ab99485a6
Fix lint error 2019-07-21 23:32:54 +08:00
NIC619
41141c028b
FIx:
check topic exist in `pubsub.peer_topics`
2019-07-21 23:16:41 +08:00
NIC619
fd1f318b0c
Fix:
in mesh heartbeat, select from gossipsub peers subscribed to the topic
2019-07-21 23:14:08 +08:00
NIC619
404dc67e83
Fix:
prevent selecting peers from topic not in peer topics
2019-07-21 23:14:08 +08:00
NIC619
b5c3420c16
Refactor gossipsub.join 2019-07-21 23:14:08 +08:00
NIC Lin
1e78c21eca
Merge pull request #181 from NIC619/fix_handle_unsubscribe
Fix: handle unsubscribe message
2019-07-21 20:09:51 +08:00
mhchia
8d3b3fdb6c
Add debug msg 2019-07-20 22:47:37 +08:00
Kevin Mai-Husan Chia
085a5e7526
Apply suggestions from code review
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-20 22:43:32 +08:00
mhchia
73125b99b0
Fix the error due to the breaking change
In multiaddr, `split` is moved to `Multiaddr`'s method.
2019-07-20 22:43:31 +08:00
Kevin Mai-Husan Chia
4422888f6f
Merge pull request #180 from mhchia/fix/include-all-pkgs-setuptools
Change `packages` in `setup`
2019-07-20 22:39:57 +08:00
NIC619
36575e8c9b
Add check to prevent gossipsub re-join and re-leave 2019-07-19 19:59:48 +08:00
NIC619
89347be526
Prevent self re-unsubscription 2019-07-18 13:26:31 +08:00
NIC619
f25d97fbd3
Prevent self re-subscription 2019-07-15 16:32:05 +08:00
NIC619
6d9ec7a9c5
Handle the unsubscribe case 2019-07-15 16:28:29 +08:00
mhchia
31cd520076
Add the missing __init__.py in pb 2019-07-11 17:50:36 +08:00
mhchia
a7ce230d05
Change params multiaddr to maddr
To make pylint happy
2019-07-10 19:33:38 +08:00
mhchia
c8cb375d19
Make pylint happy
Not sure why it fails now, but happy previously.
2019-07-10 17:44:14 +08:00
mhchia
fa092505e8
Add missing "package file" 2019-07-10 00:07:58 +08:00
mhchia
10511d4930
Clean up a bit 2019-07-09 23:55:43 +08:00
NIC619
900b6d9f37
Fix docstring 2019-06-21 11:51:11 -06:00
Stuckinaboot
a4529d827d Modify peer_id str 2019-05-22 11:37:07 -04:00
Stuckinaboot
ae44c062df Remove truncating peerid when converting to string 2019-05-22 11:37:07 -04:00
ZX
49384ad3d7
Merge pull request #159 from libp2p/security-dev
Implement security
2019-05-07 09:56:19 -04:00
Stuckinaboot
867f3a70f6 Modify stream to be reader_writer 2019-05-07 01:31:54 -04:00
Robert Zajac
9052e8f8bd
The Gossipsub PR (#162)
* Add handle_rpc call to pubsub

* Scaffold gossipsub functions

* Add timer

* Implement most of mesh construction

* Implement emit and handle

* Implement fanout heartbeat

* Refactor emit

* some gossipsub cleanup and test

* minor lint stuff, more to come

* Implement publish

* Fix comment

* Modify pubsub/gossipsub so that floodsub tests pass using gossipsub router

* Add floodsub tests to gossipsub

* Handle case where select_from_minus, num_to_select > size(pool-minus)

* Add topic membership

* Implement handle ihave

* Implement most of iwant

* Add mcache.add and comments

* Refactor handle_ihave

* Implement stream write in handle_iwant

* Implement gossip heartbeat

* unresolved vars

* initial mcache code

* documenting mcache

* writing test/debugging mcache

* finished mcache test and debugged

* Make gossipsub backward compatibility its own file

* remove mcache prints

* DEBUGGING

* Add sender_peer_id to handle_rpc to get gossip test passing

* Modify gossipsub to make fanout work

* fanout maintenance test

* debugging gsub GOSSIP

* DEBUGGING

* debugged sender seen cachce

* adding lru, removing prints

* pylint cleanup

* Fix github comments in PR

* minor floodsub possible bugfix
2019-05-06 23:44:13 -04:00
Alex Haynes
eea6a9fda7
Merge pull request #157 from libp2p/router-refactor
Refactored RoutedHost into Injected Router
2019-05-05 14:45:22 -04:00
Alex Haynes
ea4fd8e81f requested changes 2019-05-05 14:32:41 -04:00
stuckinaboot
ff500e6d8e
Merge branch 'master' into security-dev 2019-05-01 17:26:23 -04:00
Stuckinaboot
144dc8d854 Move simple security to libp2p/security 2019-05-01 17:21:11 -04:00
Stuckinaboot
4333c2d061 Fix linting issues' 2019-05-01 13:54:19 -04:00
Alex Haynes
0f5854a83b
Merge pull request #160 from libp2p/providers
implement add get providers
2019-04-30 21:48:17 -04:00
Stuckinaboot
e555f17a7b Fix bug in security multistream 2019-04-30 16:07:26 -04:00
Stuckinaboot
f59f27d4d0 Integrate security selectin into libp2p system 2019-04-30 03:09:05 -04:00
Stuckinaboot
1fb4372ede Refine security 2019-04-29 18:05:38 -04:00
zixuanzh
e1d6fdae73 pass test 2019-04-28 18:18:56 -04:00
Alex Haynes
2efc0d3b5c updated peerinfo encoding stored in DHT 2019-04-28 17:38:40 -04:00
zixuanzh
8ab387ac05 refactor add_router in swarm 2019-04-28 15:00:43 -04:00
ZX
21a99287b1
Merge branch 'master' into router-refactor 2019-04-28 14:51:54 -04:00
zixuanzh
3b1a3fb198 fix tests 2019-04-28 13:58:57 -04:00
Alex Haynes
fdaa4f56b9 working on adding tests 2019-04-27 21:59:25 -04:00
Stuckinaboot
eb26661652 Remove outdated encryption folder 2019-04-27 20:09:25 -04:00
Stuckinaboot
d861a00d60 Scaffold security 2019-04-27 19:42:05 -04:00
zixuanzh
4436886371 implement add get providers 2019-04-25 13:25:09 -04:00
zixuanzh
35f75c4dcd move transport to swarm constructor 2019-04-24 22:36:09 -04:00
Alex Haynes
159301306d Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:26:17 -04:00
Alex Haynes
7962060cc0 refactored routedhost into router passed to swarm 2019-04-24 22:17:35 -04:00
Alex Haynes
6c5bac53d7 refactored routedhost into router passed to swarm 2019-04-24 22:11:54 -04:00
Alex Haynes
c4f8bf6ce1 fixed requested changes 2019-04-24 20:59:22 -04:00
Alex Haynes
73ce3461e2 fixed typo 2019-04-21 13:45:18 -04:00
Alex Haynes
9c5cd05042 updated id access in kadpeerinfo 2019-04-21 13:27:44 -04:00
Alex Haynes
5fc57c1076 removed duplicate code 2019-04-21 13:22:11 -04:00
Alex Haynes
e7424d3673 added RoutedHost and updated new_node to support it 2019-04-20 17:35:05 -04:00
Alex Haynes
b0edc47b9a removed commented out code 2019-04-19 21:04:18 -04:00
Alex Haynes
dbe3fcbf5a updated same_home_as to work for variable number of addresses 2019-04-19 21:00:55 -04:00
zixuanzh
69f9aa5f0e refactor KadPeerInfo construction 2019-04-19 20:44:17 -04:00
zixuanzh
1512ae27a1 fix all tests 2019-04-19 20:09:32 -04:00
zixuanzh
fb687dad09 refactor peerinfo 2019-04-19 20:09:32 -04:00
zixuanzh
c03f2f63d2 replace id with peer_id 2019-04-19 20:09:32 -04:00
zixuanzh
9ddbd18ded replace node with KadPeerInfo
replace node with kadpeerinfo
2019-04-19 20:09:18 -04:00
ZX
14da7d49fe
Merge branch 'master' into peer_routing 2019-04-17 21:22:37 -04:00
Alex Haynes
d5c7cc7cb7 added changes from future commits to last passing bmuller commit 2019-04-17 20:21:59 -04:00
zixuanzh
d04798ce7c lru cache seen_messages 2019-04-05 21:46:03 -04:00
Stuckinaboot
686c55b09c Remove unnecessary print #146 2019-04-05 17:38:29 -04:00
Stuckinaboot
feaa393c5f Fix seen messages bug 2019-04-03 23:32:03 -04:00
Stuckinaboot
9d16aa834d Modify pubsub to have seen message check incorporate seqno and node id 2019-04-03 23:32:03 -04:00
Stuckinaboot
f6299c7dee Add priority queues to handle seqno 2019-04-03 23:32:03 -04:00
zixuanzh
225bd390df add source to rpc.proto 2019-04-03 14:08:05 -04:00
zixuanzh
0238dff217 remove unused code 2019-04-02 21:17:48 -04:00
zixuanzh
41d1aae55b clean up 2019-04-01 16:55:44 -04:00
zixuanzh
6eb070b78e fix all tests 2019-04-01 16:23:20 -04:00
zixuanzh
89a19a9213 reworked floodsub logic 2019-03-31 22:16:28 -04:00
zixuanzh
971dbe1a96 fix encoding issue 2019-03-30 19:30:58 -04:00
zixuanzh
f5af4b9016 remove message.py 2019-03-30 18:49:50 -04:00
zixuanzh
aec783b843 reworked subscribe unsubsrcibe 2019-03-30 17:59:08 -04:00
Alex Haynes
bf17f424b3 RPC conversion progress 2019-03-29 16:23:30 -04:00
zixuanzh
3cfeccaf17 rewrote get_hello_packet 2019-03-28 15:25:33 -04:00
zixuanzh
81d121a029 update from to from_id in proto 2019-03-28 15:23:56 -04:00
zixuanzh
1e8d93fcf6 add generated rpc code 2019-03-28 09:55:14 -04:00
zixuanzh
041e0fbe34 add pubsub proto 2019-03-28 09:41:38 -04:00
Christophe de Carvalho
c5289952ee
add routing interfaces 2019-03-26 20:44:01 +01:00
Christophe de Carvalho
2d1b9a03d1
clean up 2019-03-26 20:20:14 +01:00
Christophe de Carvalho
cd8cb5c443
refactoring of the code to implement IAdvertiser and IDiscoverer 2019-03-26 20:16:16 +01:00
Christophe de Carvalho Pereira Martins
4889a0a790
First POC of peer routing using kademlia lib 2019-03-26 20:15:19 +01:00
Christophe de Carvalho Pereira Martins
c24a279d2d
update kadmelia lib 2019-03-26 20:14:42 +01:00
stuckinaboot
57077cd3b4 [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-03-23 13:52:02 -04:00
zixuanzh
d67aeb1974 fix linting issues 2019-03-17 21:30:56 -04:00
zixuanzh
fea26d9087 refactor new_node 2019-03-17 19:20:46 -04:00
Stuckinaboot
a5fa053815 Add return value to Notify 2019-03-15 11:22:20 -04:00
Stuckinaboot
01099e9f2c Add opened_stream call for non-initiator 2019-03-15 11:22:20 -04:00
Stuckinaboot
745c5e677c Fix outstanding unrelated lint issue in multiselect_client 2019-03-15 11:22:20 -04:00
Stuckinaboot
58bdee499b Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot
22d6c97ef3 Add conn to net stream and conn tests 2019-03-15 11:22:20 -04:00
Stuckinaboot
670076035c Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot
3e64520506 Make notifee functions all async 2019-03-15 11:22:20 -04:00
Stuckinaboot
7a8878411f Add tests for notify 2019-03-15 11:22:20 -04:00
Stuckinaboot
1ce8225c7d Implement notify feature 2019-03-15 11:22:20 -04:00
Stuckinaboot
f7d1bb3a89 Add notify function to network interface 2019-03-15 11:22:20 -04:00
Stuckinaboot
cdfcdd528e Add notifee interface 2019-03-15 11:22:20 -04:00
Hyungsuk Kang
78d0a658bb Typo fixes (#127)
* Update transport_interface.py

* Update tcp.py

* Update README.md
2019-02-24 23:11:54 -05:00
Robert Zajac
82840b5e6c
Stream rearchitecture (#126)
* Add generic protocol handler

* Add generic protocol handler to stream muxing pipeline

* Modify conn_handler to only deal with connections

* mplex accept stream architecture changes

* Add create generic protocol handler

* Fix minor bugs

* who would win 4 devs or one not

* Debugging

* rearch with handle_incoming infinite loop, seems to work, needs cleanup"

* passing linting, still needs cleanup

* fixing linting again; code still needs cleanup

* fixing tests; code still needs cleanup

* adding test cleanup and task cleanup, removing prints

* linting, and cleanup complete

* storing connections based on peer id

* remove dead code

* remove unnecessary peer_id
2019-02-24 20:58:23 -05:00
Alex Haynes
17c778de15
Peer Discovery Interface (#123)
* added file

* basic interface modeled on go repo

* fixed linting

* updated based on comments
2019-02-24 18:37:56 -05:00
Christophe de Carvalho
9b6d61ca63 remove unused multiaddr module (#110) 2019-01-30 19:41:03 -05:00
Alex Haynes
7736d2afd2
updated reading to read until you see a message for your stream (#100)
* updated reading to read until you see a message for your stream

* added timeout to decode uvarint

* resolved comments

* shortened long line
2019-01-28 16:15:22 -05:00
zixuanzh
71a6b868e1 fix merge conflicts 2019-01-10 20:45:03 +08:00
Alexander Koshkin
113021eacd Delete trailing whitespace issue, ignore previously untested folders for now. 2019-01-09 22:30:18 +03:00
Alexander Koshkin
ce6ddb27a9 Migrate to new project structure. 2019-01-09 21:38:56 +03:00
Christophe de Carvalho
611de28aca Add basic support for multiaddr addresses and improvement around peer id (#75)
* Improved peer ID construction and usage

* peer id object is directly passed to the network

no need to cast from a string to an ID

* don't base64 encode the peer id when loading from public key

* use proper multiaddr address

- keep multiaddr object into peerstore instead of string
- update network code to use new multiaddr lib
- update tests and example

* don't instanciate peerstore object in constructor

This has side effect where the same peerstore
is used for different instance of Libp2p

* add connect method to basic_host

* use zaibon's fork of sbuss/py-multiaddr

* lint
2018-11-29 10:06:40 -05:00
zixuanzh
ff6fdccea4 refactor libp2p 2018-11-25 01:45:13 -05:00
zixuanzh
c5c9d3e5c9 lint 2018-11-12 13:02:49 -05:00
Robert Zajac
61e11a2716 adding additional bugfixed files 2018-11-12 11:00:43 -05:00
Robert Zajac
e2b7ad8926 minor fix 2018-11-11 20:46:32 -05:00
zixuanzh
64002acc64 implement new_node 2018-11-11 20:41:21 -05:00
Robert Zajac
f54a3c52d8 WIP swarm listen 2018-11-11 20:29:55 -05:00
Robert Zajac
684cb90908 WIP swarm listen 2018-11-11 20:29:17 -05:00
Alex Haynes
cca13cbfea made idOpt optional 2018-11-11 19:46:00 -05:00
Alex Haynes
1e2fc19f9c
Merge branch 'master' into rsa-public-key 2018-11-11 13:23:34 -05:00
Alex Haynes
6c2868cb3a removed space 2018-11-11 13:22:31 -05:00
Alex Haynes
e20d6df667 added rsa public key generation 2018-11-11 13:21:47 -05:00
Robert Zajac
4681e1e08d libp2p end to end test 2018-11-11 12:36:15 -05:00
zixuanzh
af6784546e entry point wip 2018-11-11 10:43:03 -05:00
zixuanzh
2bde260f5f refactor for sprint 2018-11-11 09:56:44 -05:00
zixuanzh
5f9c3026aa restructured 2018-10-31 19:56:24 +01:00