StarMirror/py-libp2p
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,469 Commits 16 Branches 6 Tags 3.1 MiB
0324a69841
Commit Graph

708 Commits

Author SHA1 Message Date
NIC619
68573e94d3
Have StreamError inherit from IOException 2019-09-15 16:34:16 +08:00
NIC619
f368f5e93b
Apply PR feedback 2019-09-15 15:09:58 +08:00
NIC619
501513b747
Update IMultiselectClient 2019-09-14 22:24:53 +08:00
NIC619
9bad7a61f0
Add some loggings to pubsub 2019-09-14 21:54:26 +08:00
NIC619
786a03544c
Add some loggings to swarm and cosmetic updates 2019-09-14 21:47:49 +08:00
NIC619
65a48d5c51
Remove unused select_protocol_or_fail 2019-09-14 17:42:18 +08:00
mhchia
8c8c206c33
flake8 2019-09-11 18:33:52 +08:00
mhchia
451f993058
Fix isort 2019-09-11 18:05:41 +08:00
mhchia
c1ffc0ab07
Fix transport.dial in swarm 2019-09-11 17:13:21 +08:00
Alex Stokes
cb1a25f94c
Execute the todo to remove an unused argument 2019-09-10 21:01:03 -04:00
Alex Stokes
4bd32cc4bc
Add logs during connection handshake 2019-09-10 19:02:29 -04:00
Alex Stokes
f128c746f0
Write data payload as hex to log 2019-09-10 16:17:40 -04:00
Alex Stokes
68e75707e4
Enhance logs 2019-09-10 16:04:18 -04:00
mhchia
31fb4e0b69
Rewrite _wait_for_data, to handle task precisely 
Make the futures first, and then we can compare them with the return
value from `asyncio.wait`.
 2019-09-10 23:38:45 +08:00
mhchia
df87f5adb9
Add tests against the daemon for close/reset 2019-09-10 18:01:16 +08:00
mhchia
e5eb01d22b
Fix stream read 2019-09-10 18:01:16 +08:00
mhchia
df312f3e57
Fix linting 2019-09-10 18:01:15 +08:00
mhchia
be2c0f122a
Fix close behavior 2019-09-10 18:01:14 +08:00
Brian Cloutier
b2146c5268
Don't crash on large messages 2019-09-09 17:13:26 -04:00
Alex Stokes
155bec0562
Fix initiator flag during secio upgrade 2019-09-09 16:40:14 -04:00
Alex Stokes
2025a5c7f1
Restore initiator flag to BaseSession type 2019-09-08 15:40:02 -04:00
Alex Stokes
50ae439d20
remove friendly IDs 2019-09-07 11:04:20 -04:00
mhchia
6c1f77dc1a
Fix: Change the event.close to event.set 
And add missing parts.
 2019-09-06 21:35:15 +08:00
mhchia
1cd969a2d5
Fix: Add typing in functions 2019-09-06 20:02:35 +08:00
mhchia
649a230776
Fix MplexStream.read 2019-09-06 17:26:40 +08:00
mhchia
95926b7376
Temp for mplex_stream 2019-09-06 01:08:42 +08:00
mhchia
207fa75d8f
Add reset and close 2019-09-05 23:44:22 +08:00
mhchia
10415cb956
Use ReadWriteCloser for conns and streams 2019-09-05 23:24:17 +08:00
mhchia
eac159c527
Restructure mplex and mplex_stream 2019-09-05 22:29:33 +08:00
mhchia
96230758e4
Add events in MplexStream 
And modify a little bit of `close` and `reset`
 2019-09-05 18:18:39 +08:00
Alex Stokes
451ec2664a
Address incorrect typing in pycryptodome dependency 2019-09-04 10:19:27 -07:00
Alex Stokes
a099b9c65d
Clean up temporary hold over from previous PR 2019-09-04 10:15:30 -07:00
Alex Stokes
b214f88f75
Avoid using message-based IO in the plaintext protocol 
Can reuse the machinery in `secio` but need to generalize the
"buffering" there
 2019-09-04 10:12:43 -07:00
Alex Stokes
f86ba7283d
Implement signing for RSA 
- mainly for use in `secio` w/ RSA-based identities b/t peers
 2019-09-04 10:12:43 -07:00
Alex Stokes
25f504ad35
Allow RSA public key type when deserializing keys 2019-09-04 10:12:43 -07:00
Alex Stokes
1a359770dd
Use msgio IO and proper buffering in secio implementation 2019-09-04 10:12:43 -07:00
Alex Stokes
fc3e3a4be5
Changes to type hints to match new abstractions 2019-09-04 10:12:43 -07:00
Alex Stokes
6d97702da7
Merge pull request #277 from ralexstokes/add-io-abstractions 
Introduces IO abstractions and classes for `msgio` IO
 2019-09-04 19:11:50 +02:00
Alex Stokes
a764fd4e6f
simplify some of the msgio class hierarchy via PR feedback 2019-09-04 09:57:04 -07:00
Kevin Mai-Husan Chia
e9ca372fb5
Merge pull request #271 from mhchia/fix/pubsub-interop 
Pubsub interop with go-libp2p-daemon
 2019-09-04 22:28:14 +08:00
mhchia
34b489af25
Fix kad_peerinfo according to peerinfo 2019-09-04 21:37:33 +08:00
mhchia
0e3d4508d6
PR feedback 
- Use `Sequence` instead of `List`
- Add note
- Remove redundant words in docstring
 2019-09-04 20:52:18 +08:00
Kevin Mai-Husan Chia
5113785543
Update libp2p/pubsub/pubsub.py 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-09-04 20:32:43 +08:00
mhchia
677531db76
Fix pubsub tests 2019-09-04 15:33:07 +08:00
Alex Stokes
2a02f92f77
Temporary porcelain until next PR 2019-09-03 21:59:50 -07:00
Alex Stokes
8e8318aa5c
Introduces IO abstractions apart from asyncio or those attached to IRawConnection 
Also adds `msgio` utilities to mirror the Go implementation
 2019-09-03 21:59:50 -07:00
Alex Stokes
cd3e093001
remove leftover simple security module 2019-09-03 21:38:33 -07:00
Alex Stokes
506093216c
Merge pull request #276 from ralexstokes/fix-secio-impl 
Fix secio impl
 2019-09-03 23:01:45 +02:00
Alex Stokes
272ab60d47
Merge pull request #275 from ralexstokes/use-different-ecc-backend 
Use a different ECC backend `fastecdsa` with a compatible serializer
 2019-09-03 23:00:33 +02:00
Alex Stokes
30456f8018
Use a different ECC backend with a compatible serializer 
This library has the ``SEC1`` encoder which is compatible
with the serialization of ECC keys/points used in the Go libp2p impl
 2019-09-03 13:44:25 -07:00
Alex Stokes
bfd674e22c
Try all pairs of choices, not just a small subset via zip 2019-09-03 13:28:31 -07:00
Alex Stokes
f8bbaf60a1
Add more information to the peer mismatch exception 2019-09-03 13:28:18 -07:00
Alex Stokes
350cc04c97
Allow the key pair to drive ID generation 2019-09-03 13:26:58 -07:00
Alex Stokes
345e696a7d
Add option to inline "short" public keys for peer IDs 
Added to remain interoperable w/ the Go implementation
 2019-09-03 13:14:04 -07:00
mhchia
961e51fa2e
Remove leftover prints 2019-09-03 23:39:29 +08:00
mhchia
7f20ab781d
Fix gosssipsub tests 2019-09-03 23:37:34 +08:00
mhchia
4f7bb30d82
Add INetStream to type StreamReader 
TODO: Make stream readers implement `Reader`
 2019-09-03 22:59:44 +08:00
mhchia
7385a7a677
Add is_gossipsub fixture in interop test 
To use the same code to test against both routers: floodsub and
gossipsub.
 2019-09-03 16:49:00 +08:00
mhchia
33dae87c35
Add pubsub test for gossipsub 2019-09-03 16:07:44 +08:00
mhchia
fd1f466002
Fix: failed to open stream using existing conn 
Fix #233
 2019-09-03 14:12:16 +08:00
mhchia
194b494057
Tested against subscriptions and publish 2019-09-02 23:21:57 +08:00
mhchia
3717dc9adf
Add helper functions 2019-09-02 21:01:13 +08:00
mhchia
dfd9ebdc5e
Change PeerInfo to remove dep on PeerData 2019-09-02 14:30:27 +08:00
mhchia
1b5d064a8d
Add utility functions for libp2p bindings 
To prepare for pubsub interop test
 2019-09-02 14:30:25 +08:00
mhchia
77a0cc3a87
Add the missing __init__.py 2019-09-02 14:18:52 +08:00
mhchia
aa0866698f
PR feedback: Add check in Swarm.close_peer 2019-08-31 22:37:59 +08:00
Kevin Mai-Husan Chia
1e59438f25
Update libp2p/network/swarm.py 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-08-31 22:32:32 +08:00
mhchia
b2c5371323
Add TODO for Swarm.connections 2019-08-29 22:00:07 +08:00
mhchia
c61a06706a
Refactor interop tests and factories 
- Add `close` and `disconnect` in `Host`
- Add `close` and `close_peer` in `Network`
- Change `IListener.close` to async, to await for server's closing
- Add factories for security transports, and modify `HostFactory`
 2019-08-29 21:38:06 +08:00
mhchia
34a4d7b0ed
Add the missing StreamID class 2019-08-28 21:45:18 +08:00
mhchia
d35b8ffc64
Conform stream_id to go-mplex 2019-08-28 21:43:34 +08:00
Alex Stokes
9b60e1757d
Merge pull request #269 from ralexstokes/add-reset-to-network-stream 
Expose `reset` method on `NetStream`
 2019-08-26 21:27:57 +02:00
Alex Stokes
a193ae81fd
Fix typo w/ header tag under different reset scenarios 2019-08-26 11:39:30 -07:00
Alex Stokes
c5eda4065d
Expose reset method on NetStream 2019-08-26 11:38:39 -07:00
Alex Stokes
98a0e76dda
Merge pull request #268 from mhchia/fix/mplex-interop 
Fix: name of a `MplexStream` is not handled in `Mplex`
 2019-08-26 19:51:57 +02:00
Alex Stokes
66c3bacf7e
Merge pull request #254 from ralexstokes/implement-secio 
Implement `secio`
 2019-08-26 19:22:00 +02:00
Alex Stokes
fa0acd9fc5
Apply PR feedback 2019-08-26 10:03:12 -07:00
Alex Stokes
c1ffb03f77
Update comment to reflect correct function 2019-08-26 09:51:49 -07:00
mhchia
b6c8ab0dc9
Fix #259: Use the unsigned LEB128 impl from py-wasm 
Reference: https://github.com/ethereum/py-wasm/blob/master/wasm/parsers/leb128.py
 2019-08-26 20:41:10 +08:00
mhchia
d59870ebbf
Fix MplexStream error 
When receiving a `NewStream`, the message of that packet is the
name of the stream, which should be handled, rather than letting it go
into the message queue.
 2019-08-26 20:39:08 +08:00
NIC619
5e68aff1d1
Fix conn attribute and docstring in mplex 2019-08-25 14:42:44 +08:00
Alex Stokes
a363ba97d1
Work in terms of the IRawConnection abstraction 2019-08-24 22:52:09 +02:00
Alex Stokes
eb5ef39399
Convert message IO to work w/ a RawConnection. 2019-08-24 22:47:56 +02:00
Alex Stokes
7c004a4e14
Mypy fixes 2019-08-24 22:38:26 +02:00
Alex Stokes
44e5de636f
Add "friendly" peer ID string representation for debugging 2019-08-24 22:38:26 +02:00
Alex Stokes
852609c85d
Clean up base session type 2019-08-24 22:38:26 +02:00
Alex Stokes
b8c0ef9ebb
Fix bugs in secio implementation 2019-08-24 22:38:25 +02:00
Alex Stokes
228032805a
Some code cleanup 2019-08-24 22:38:25 +02:00
Alex Stokes
3f4589d497
Get tests working 2019-08-24 22:38:25 +02:00
Alex Stokes
d176115972
Add secio to security upgrader suite 2019-08-24 22:37:46 +02:00
Alex Stokes
376a5d4fc6
Adjust callsite 2019-08-24 22:37:45 +02:00
Alex Stokes
1adef05e94
Typing and linting fixes 2019-08-24 22:37:45 +02:00
Alex Stokes
0fa3331b8c
Add clearer indication of "self encryption" 2019-08-24 22:37:45 +02:00
Alex Stokes
8e913a3faa
Dispatch serialization of keys based on key type 
- Add some tests to check high-level roundtrip
 2019-08-24 22:37:45 +02:00
Alex Stokes
4d30b31c55
Finish first pass at secio implementation 2019-08-24 22:37:45 +02:00
Alex Stokes
af2e50aaf4
Add facility for authenticated encryption 2019-08-24 22:37:45 +02:00
Alex Stokes
228c17ae9e
Add ECDH key exchange utility 2019-08-24 22:37:44 +02:00
Alex Stokes
fb13dfa7b3
Add sign and verify operations for secp256k1 keys 2019-08-24 22:37:44 +02:00
Alex Stokes
3c97a5a0ed
Add ECC key implementation 2019-08-24 22:37:44 +02:00
Alex Stokes
91e11f3ec0
[wip] more secio implementation 2019-08-24 22:37:44 +02:00
Alex Stokes
0cc3fc24a7
Add source for some secure bytes, e.g. to provide a nonce 2019-08-24 22:37:44 +02:00
Alex Stokes
b59c5d6ca1
Add "msgio" functions 2019-08-24 22:37:44 +02:00
Alex Stokes
26165b0729
[wip] sketch of secio handshake 2019-08-24 22:37:44 +02:00
Alex Stokes
fd08bcf624
Add secio protobufs 2019-08-24 22:37:43 +02:00
Alex Stokes
23f53ef954
Allow optional peer ID in a security session 2019-08-24 22:37:43 +02:00
Alex Stokes
27c0a4f77b
formatting 2019-08-24 22:37:43 +02:00
Alex Stokes
5b32bfdd3f
Add call to wait_closed method of asyncio.StreamWriter 2019-08-24 22:06:24 +02:00
Alex Stokes
9c5fb4fa5a
Encapsulate concept of a "stream id" to a "muxed" connection 2019-08-24 21:50:07 +02:00
Alex Stokes
e29c1507bf
remove unused fields 2019-08-24 21:50:06 +02:00
Alex Stokes
73495038e1
remove simple security 2019-08-24 21:39:25 +02:00
Alex Stokes
1790e48c99
Remove deprecated file 2019-08-24 21:36:30 +02:00
mhchia
b516579256
Remove the unnecessary RSAPrivateKey.from_bytes 2019-08-22 22:54:14 +08:00
mhchia
c1eacf221f
PR feedback 
- Check if the received peer id matches the one we initialize the
session with.
- Move the check inside `run_handshake`
 2019-08-22 22:53:49 +08:00
Kevin Mai-Husan Chia
7c630df610
Update libp2p/security/insecure/transport.py 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-08-22 22:53:49 +08:00
mhchia
16a4fd33c1
PR feedbacks 
- Move exceptions to exceptions.py
- Raise `UpgradeFailure` in upgrader
- Refine the try/catch for upgraders in swarm
 2019-08-22 22:53:49 +08:00
mhchia
3e04480d62
Raise HandshakeFailure in transport 
Change the exception handling flow.
Raise `SecurityUpgradeFailure` in security_multistream.
 2019-08-22 22:53:48 +08:00
mhchia
80452d9589
Fix make_exchange_message to use the new API 2019-08-22 22:53:48 +08:00
mhchia
921bfb65cc
Verify the remote pubkey and peer_id 
- Add `from_bytes` in RSAPublicKey and Secp256k1PublicKey
- Add `pubkey_from_protobuf` to parse pubkey from protobuf
- Verify key and peer_id in `InsecureSession.run_handshake`
 2019-08-22 22:53:48 +08:00
mhchia
ef476e555b
Use RawConnection.read 
Instead of accessing its reader and writer directly.

TODO: considering add `ReaderWriterCloser` interface and let connection
and stream inherit from it.
 2019-08-22 22:53:48 +08:00
mhchia
0b466ddc86
Add lock to RawConnection 
To avoid `self.writer.drain()` is called in parallel.
Reference: https://bugs.python.org/issue29930
 2019-08-22 22:53:47 +08:00
mhchia
5768daa9bf
PR feedbacks 
- Nits
- Add `SecurityUpgradeFailure` and handle `UpgradeFailure` in Swarm.
 2019-08-22 22:53:47 +08:00
Kevin Mai-Husan Chia
2a1367b011
Apply suggestions from code review 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-08-22 22:53:47 +08:00
mhchia
de8d356955
Fix tests failure due to lack of peer id 
Fix it through doing plaintext handshake.
 2019-08-22 22:53:47 +08:00
mhchia
bb7d37fd4f
Fix msg encoding 
- Change varint-prefix encode to fixedint-prefix(4 bytes) encode.
 2019-08-22 22:53:46 +08:00
mhchia
22b1a5395d
A working plaintext 2.0 without validation 2019-08-22 22:53:46 +08:00
mhchia
a0923d202a
Move varint and delim read/write to toplevel 
To `libp2p.utils`.
 2019-08-22 22:53:46 +08:00
mhchia
5192944724
Update pb 2019-08-22 22:53:46 +08:00
mhchia
59b373b48a
Add plaintext.proto 
Update Makefile to handle the import in `plaintext.proto`.
Import path is modified to be relative to the project root.
And we run `protoc` from where `Makefile` locates, i.e. the project
root.

Reference:
- plaintext.proto: 62b2c6c482/sec/insecure/pb/plaintext.proto
 2019-08-22 22:53:46 +08:00
mhchia
7bc363f2fa
Remove initiator in Mplex 
Besides, fix the wrong passed `multi_addr` to `mplex_stream`.
 2019-08-22 22:53:45 +08:00
mhchia
2752e7a04b
Ignore mypy error here 2019-08-22 15:58:38 +08:00
mhchia
dbd4ed44dd
Fix the issue when importing from other package 2019-08-22 15:36:17 +08:00
mhchia
550c23f9f9
PR feedback 
- Use the order in `MuxerMultistream` as the precedence in multiselect
 2019-08-21 11:43:25 +08:00
mhchia
8596f7390f
PR feedback: set protocol_id to constants 2019-08-21 11:43:24 +08:00
mhchia
d7d8440b2c
PR feedback: nitpicks 2019-08-21 11:43:24 +08:00
mhchia
4358a4bc89
Negotiate multiselect version for Muxer 
`MuxerMultistream` is introduced to negotiate `Multiselect` version
before negotiating Multiplexer's version. This is required by
multistream 1.x
 2019-08-21 11:43:24 +08:00
Alex Stokes
e1d3f1601f
Satisfy mypy 2019-08-20 19:28:32 +02:00
Alex Stokes
87d943aa39
Internalize the protobuf serialization to the concept of a Key 
Given its use across various components of `libp2p` (not just peer IDs),
it makes the abstraction cleaner to pull the serialization into the
key class and expose the canonical serialization to bytes.
 2019-08-20 19:01:36 +02:00
mhchia
5d611801c7 Fix isort 2019-08-18 19:51:04 +08:00
mhchia
86d4ce1da8 Add delim_encode and delim_read 
- Add `StreamCommunicator` and `RawConnectionCommunicator`, read/write
messages with delim codec, with `IMuxedStream` and `IRawConnection`
respectively.
- Use it in `Multiselect` and `MultiselectClient`.
 2019-08-18 19:51:04 +08:00
mhchia
8cd23abfe2 Remove the leftover merge related stuff 2019-08-16 11:03:16 +08:00
mhchia
e293b89925 Fix mypy 
It seems the stub doesn't allow default value for params
 2019-08-16 11:03:16 +08:00
mhchia
c5f32bf431 PR feedback for MplexStream.read 2019-08-16 11:03:16 +08:00
Kevin Mai-Husan Chia
8699568d43 Update libp2p/stream_muxer/mplex/mplex_stream.py 
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
 2019-08-16 11:03:16 +08:00
mhchia
f2c31f6fe3 Fix isort 2019-08-16 11:03:16 +08:00
mhchia
b27cd0f24f Use bytearray over bytes 
To avoid copies.
 2019-08-16 11:03:16 +08:00
mhchia
92320523d5 Add the missing exceptions.py 2019-08-16 11:03:16 +08:00
mhchia
e37b8bcf19 mypy: Add read_buffer_nonblocking in Mplex 2019-08-16 11:03:16 +08:00
mhchia
f281e3e1db flake8 2019-08-16 11:03:16 +08:00
mhchia
9f8276fa84 Support read(n=-1) 
Now, `n=-1` indicates that we want to read until EOF. However, now we
only read until we have no new message.
 2019-08-16 11:03:16 +08:00
mhchia
9cb6ec1c48 Modify the behavior of MplexStream.read 2019-08-16 11:03:16 +08:00
mhchia
2485a00e24 Modify NetStream to read n bytes 2019-08-16 11:03:16 +08:00
Alex Stokes
7535a02da7
Clean up key gen 2019-08-15 19:24:30 -07:00
Alex Stokes
9a4e23a803
mypy protobuf plugin requires keyword-based initializers 2019-08-15 16:36:32 -07:00
Alex Stokes
82bae341a7
Run isort over files that were missing it 2019-08-15 16:33:35 -07:00
Alex Stokes
2e3ffb9d53
Use types for {Private,Public}Key and address other missing type hints 2019-08-15 16:33:34 -07:00
Alex Stokes
e7d2681fc0
Move base implementations into BaseSession 2019-08-15 16:33:34 -07:00
Alex Stokes
9e18d7561d
Supply local priv and pub key when upgrading to a secure transport 2019-08-15 16:33:34 -07:00
Alex Stokes
7942b7eaa7
Expose writer 2019-08-15 16:33:34 -07:00
Alex Stokes
20dd7d777a
More efficiently remove trailing newline from message 2019-08-15 16:33:34 -07:00
Alex Stokes
0ebc8ffb21
Wire some missing properties up 2019-08-15 16:33:34 -07:00
Alex Stokes
02e073d85a
Keep the host's private key for use in transports 2019-08-15 16:33:33 -07:00
Alex Stokes
879cbf1abd
Add an "insecure session" that satisfies the ISecureConn interface 2019-08-15 16:33:33 -07:00
Alex Stokes
1e5357a1e1
Update the ISecureConn interface following the reference and simplify accordingly 2019-08-15 16:33:33 -07:00
Alex Stokes
ab7653526f
Code cleanup / formatting 2019-08-15 16:33:33 -07:00
Alex Stokes
b98025c379
Move security transports into their respective sub-packages 2019-08-15 16:33:33 -07:00
Alex Stokes
d50e1b6872
Use direct types over indirect types 2019-08-15 16:33:32 -07:00
Alex Stokes
ff5eaf2429
Merge pull request #249 from ralexstokes/fix-mypy-protos 
Add `mypy` protobufs plugin and regenerate protobufs
 2019-08-15 16:15:26 -07:00
Alex Stokes
0f81ca42a6
Add mypy protobufs plugin and regenerate protobufs 2019-08-15 16:01:44 -07:00
mhchia
97308c897e
Add the missing __init__.py in pb/ 
Error: https://circleci.com/gh/mhchia/trinity/5342?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
 2019-08-15 21:03:45 +08:00
Alex Stokes
125c5d8e2c
Adjust inheritance so that the MRO is clear for the chain of keys 2019-08-14 09:30:23 -07:00
Alex Stokes
3b19104284
Add missing ABC declaration following PR feedback in #240. 2019-08-14 09:25:54 -07:00
Alex Stokes
08e6f2a30c
Update libp2p/crypto/secp256k1.py 
pass the secret on to `coincurve` lib

Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-08-14 09:02:06 -07:00
Alex Stokes
53e583a068
Update libp2p/crypto/secp256k1.py 
Pass the secret on to `coincurve` lib

Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-08-14 09:02:06 -07:00
Alex Stokes
ad20d8cb00
Make a KeyPair dataclass for passing around key pairs 2019-08-14 09:00:40 -07:00
Alex Stokes
6506079a36
Generate peer IDs according to the spec 2019-08-14 09:00:39 -07:00
Alex Stokes
f9e859a8e9
Clean up peer.ID 2019-08-14 09:00:39 -07:00
Alex Stokes
c40314a043
Use new RSA key type 2019-08-14 09:00:39 -07:00
Alex Stokes
61f78c8feb
Add abstraction for a cryptographic key 2019-08-14 09:00:39 -07:00
Alex Stokes
992585852f
Use explicit imports in top-level declaration 2019-08-13 17:43:21 -07:00
Alex Stokes
3debd2c808
Run black and isort w/ the new config 2019-08-13 14:36:42 -07:00
Chih Cheng Liang
28f6de37ee
Fix the rest of the typing hints (#232) 
* ignore kad

* fix swarm, and minor

* fix init and swarm

* ignore pb

* enable mypy

* fix basic host

* fix tcp

* fix mplex

* add typing for pb

* skip format pyi

* [mypy] no need to ignore pb now

* add typing to chat
 2019-08-11 16:47:54 +08:00
Chih Cheng Liang
dbb702548f add NegotiableTransport type 2019-08-08 16:09:02 +08:00
Chih Cheng Liang
9851ee01fb sort import 2019-08-08 16:09:02 +08:00
Chih Cheng Liang
c536aa3e07 flake8 2019-08-08 16:09:02 +08:00
Chih Cheng Liang
5903012e0e add typing to protocol_muxer 2019-08-08 16:09:02 +08:00
NIC Lin
b26426214e
Update libp2p/pubsub/pubsub.py 
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
 2019-08-07 11:43:32 +08:00
NIC619
d4febea469
Message was not enforced to carry signature yet 2019-08-06 13:05:31 +08:00
NIC619
1cea1264a4
Raise exception when topic validation failed 2019-08-06 12:38:31 +08:00
NIC619
9a1e5fe813
Add ValidationError 2019-08-06 12:37:34 +08:00
NIC619
47643a67c6
Apply PR feedback 2019-08-06 12:32:18 +08:00
NIC619
b96ef0e6c7
Fix: 
`_is_subscribed_to_msg` need only subscribe to one of the topics
 2019-08-05 18:20:04 +08:00
NIC619
a2efd03dfa
Schedule push_msg into a task 2019-08-05 18:19:32 +08:00
NIC619
2bb7f42c20
Add validators to push_msg 2019-08-05 18:19:32 +08:00
NIC619
19ce5bb420
Add signature_validator stub and docstring 2019-08-05 18:19:32 +08:00
NIC619
e1b86904e3
Add validate_msg and test 2019-08-05 18:19:32 +08:00
NIC619
ec2c566e5a
Fix validator return type and add docstring 2019-08-05 18:19:32 +08:00
NIC619
f8ca4fa1ef
Add get_msg_validators and test 2019-08-05 18:19:32 +08:00
NIC619
cf69f7e800
Rename to set_topic_validator and add test 2019-08-05 18:19:31 +08:00
NIC619
b1f4813195
Add add/remove topic validator functions 2019-08-05 18:19:31 +08:00
NIC619
3973f1d13c
Add pubsub.topic_validators 2019-08-05 18:19:31 +08:00
Chih Cheng Liang
cb3a59e0ac
ttl as int 2019-08-05 17:02:18 +08:00
Chih Cheng Liang
63c733c3f5
PR feedback 2019-08-05 16:58:34 +08:00
Chih Cheng Liang
ccfb6eb35f
remove constructor of TCP 2019-08-05 16:56:56 +08:00
Chih Cheng Liang
e763f57930
run isort 2019-08-05 11:47:23 +08:00
Chih Cheng Liang
7a04ebb51f
run black 2019-08-05 11:21:20 +08:00
Chih Cheng Liang
9e0a806218
move stream and connection interfaces to abc 2019-08-05 11:17:38 +08:00
Chih Cheng Liang
c804f5ad19
minor 2019-08-05 10:47:59 +08:00
Chih Cheng Liang
29091266fc
add still needed TYPE_CHECK 2019-08-05 10:46:49 +08:00
Chih Cheng Liang
87ef2e4618
remove if TYPE_CHECKING as much as possible 2019-08-05 10:46:49 +08:00
Chih Cheng Liang
4c9a930f84
stream_muxer done 2019-08-05 10:45:47 +08:00
Chih Cheng Liang
dadac423f2
typed muxed_connection_interface.py 2019-08-05 10:45:47 +08:00
Chih Cheng Liang
b64ed9fd6f
typed mplex.utils 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
239a5c88fb
add typing to mplex 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
36b7e8ded9
Refactor HeaderTags 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
29fbb9e40a
add typing to transport 2019-08-05 10:42:43 +08:00
Alex Stokes
7477b29508 run black w/ extended line length 2019-08-04 12:37:41 +08:00
Alex Stokes
cb301fcc51 Opt-out of linting on the special cases we have, given new ignore rules 2019-08-04 12:37:27 +08:00
Alex Stokes
69ec86c871
Remove grpc dependency 2019-08-03 11:17:28 -07:00
mhchia
0a5b4a88ca
Fix flake8 for the existing code 2019-08-04 00:18:30 +08:00
Alex Stokes
c8005c8113 Run isort in repo 2019-08-03 17:50:14 +08:00
Alex Stokes
bd8d45fbc1
Refactor ID to not use third-party type for cryptographic keys 
Remove `ID.from_privkey` which would require specific knowledge per cryptosystem
 2019-08-02 21:51:16 -07:00
Alex Stokes
28e1a03dc4
remove empty yamux impl 2019-08-02 15:03:59 -07:00
Alex Stokes
e55d9f2e60
restore generated protobuf files 2019-08-02 12:09:27 -07:00
Alex Stokes
88a3a3159e
Add clarifying comment about InsecureTransport 2019-08-02 12:07:35 -07:00
Alex Stokes
7b7c8ad30d
run black over dangling files 2019-08-02 11:47:08 -07:00
mhchia
2e94fcf56c
Remove pylint:disable 2019-08-02 23:26:06 +08:00
NIC Lin
7a0fa7dd37
Merge pull request #196 from NIC619/fix_peer_id 
Refactor `peer.ID` class and only takes in `bytes` type argument
 2019-08-02 18:03:56 +08:00
NIC619
ee290b2ac2
Fix missing asterisk 2019-08-02 16:48:32 +08:00
NIC619
3a4b592a5a
Fix missing asterisk 2019-08-02 15:06:39 +08:00
NIC619
0173b5e0d9
Add ID.to_base58() and comparison against str type 2019-08-02 14:46:01 +08:00
NIC619
924e965537
Change argument name of Network.listen and blakc format 2019-08-02 14:45:59 +08:00
NIC619
cd684aad9e
Update peer_id to type peer.ID in pubsub folder 2019-08-02 14:45:23 +08:00
NIC619
9562cb2a46
Rename: 
`KadPeerInfo.peer_id` to `KadPeerInfo.peer_id_bytes`
 2019-08-02 14:43:01 +08:00
NIC619
f00e80bc25
Fix wrong peer id type used in KadPeerInfo 2019-08-02 14:42:10 +08:00
NIC619
b928bdb356
Convert from base58/pubkey/privkey to class method 2019-08-02 14:42:10 +08:00
NIC619
80481252ca
Refactor ID to take in type bytes only 2019-08-02 14:42:10 +08:00
Chih Cheng Liang
10a8347c6a
PR feedback 2019-08-02 14:12:59 +08:00
Chih Cheng Liang
e731f77f2d
minor 2019-08-01 19:13:43 +08:00
Chih Cheng Liang
a86f010c95
add typing to security 2019-08-01 19:12:11 +08:00
Alex Stokes
0ae9840928
Run black over repo 2019-07-31 15:00:12 -07:00
mhchia
9683d5e8ac
Add tests for Pubsub 
- `test_handle_subscription`
- `test_handle_talk`
- `test_message_all_peers`
 2019-07-31 14:15:11 +08:00
mhchia
3a42d72cd9
Fix a minor bug for pb optional field 
In `Pubsub.continuously_read_stream`, it checks whether this is a
control message enclosed in RPC message with `if rpc_incoming.control:`.
However, in pb2, the condition is always true because a default value is
returned when a field is not set. Solved it by changing it to
`if rpc_incoming.HasField("control"):`.
 2019-07-31 14:15:11 +08:00
mhchia
550289a439
Combine test_subscription.py to test_pubsub.py 
And add a bunch of tests for pubsub
 2019-07-31 14:15:10 +08:00
mhchia
96563c0d84
Add fixtures for pubsub and router 
And a starting `test_pubsub.py`
 2019-07-31 14:14:13 +08:00
NIC619
5e215901c0
Apply PR feedback 2019-07-30 23:41:28 +08:00
NIC619
437b7665c4
Fix: 
type object not subscriptable
 2019-07-30 18:00:30 +08:00
NIC619
c4105688d1
Fix after rebase 2019-07-30 17:31:08 +08:00
NIC619
e53727d301
Apply PR feedback: fix type hints 2019-07-30 16:28:25 +08:00
NIC619
2d4e23cfe2
Fix cyclic import and lint 2019-07-30 16:28:05 +08:00
NIC619
a4a0d79f6d
Improve import layout 2019-07-30 16:27:29 +08:00
NIC619
e7ac09cb94
Fix: 
Add Gossipsub attribute `peers_protocol` and do cleanup when peer removed
 2019-07-30 16:26:21 +08:00
NIC619
d716e90e17
Fix on type hints 2019-07-30 16:25:33 +08:00
NIC619
edd164c878
Add type hints to network folder 2019-07-30 16:25:17 +08:00
NIC619
e1592997a8
Add type hints to routing folder 2019-07-30 16:24:52 +08:00
NIC619
f2de986c74
Add type hints to peer folder 2019-07-30 16:24:51 +08:00
NIC619
b695b0e1ec
Add type hint to host folder 2019-07-30 16:24:34 +08:00
NIC619
b2f496d081
Fix type hints except pb msg in pubsub folder 2019-07-30 16:24:34 +08:00
NIC619
a0aa105867
Add type hint to pubsub notifee/interface 2019-07-30 16:23:15 +08:00
NIC619
3549f2ff8b
Add type hints to mcache.py 2019-07-30 16:20:48 +08:00
NIC619
63014eeaae
Add type hints to floodsub.py 2019-07-30 16:20:47 +08:00
NIC619
b920955db6
Add type hints to gossipsub.py 2019-07-30 15:39:50 +08:00
NIC619
8eb6a230ff
Fix and add type hints to pubsub.py 2019-07-30 15:32:58 +08:00
mhchia
f02d38c0ee
Reflect PR feedback 
* Rename `src` to `msg_forwarder` in pubsub/floodsub/gossipsub
* Rename Variables
* Sort imports
* Clean up
 2019-07-29 12:09:35 +08:00
mhchia
74d831d4e2
Reflect PR feedback 2019-07-28 18:06:38 +08:00
Kevin Mai-Husan Chia
70c5c84f32
Update libp2p/pubsub/floodsub.py 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-07-28 16:09:01 +08:00
Kevin Mai-Husan Chia
a1e20caebe
Update libp2p/pubsub/floodsub.py 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-07-28 16:07:11 +08:00
Kevin Mai-Husan Chia
ffb3920468
Update libp2p/pubsub/floodsub.py 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-07-28 16:06:03 +08:00
Kevin Mai-Husan Chia
c252c62009
Update libp2p/pubsub/pubsub.py 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-07-28 16:05:29 +08:00
mhchia
766d8ba1e1
A little bit clean up 2019-07-27 12:06:36 +08:00
mhchia
c028aef2de
Fix all tests 
- Dedup `perform_test_from_obj` and the test cases used in both
`test_floodsub` and `test_gossipsub_backward_compatibility.py`.
Therefore, they are put in the standalone file
`tests/pubsub/floodsub_integration_test_settings.py`. The functions
and testcases are imported from there then.
- IMO still need a refactor on the tests. There are still some duplicate
code.
 2019-07-27 11:49:03 +08:00
mhchia
65aedcb25a
Fix several tests 2019-07-26 18:35:25 +08:00
mhchia
035d08b8bd
Fix test_floodsub.py 2019-07-25 23:11:27 +08:00
mhchia
dadcf8138e
Fix the tests according to pubsub.Publish 
And refactored a bit.
 2019-07-25 16:58:00 +08:00
mhchia
cae4f34034
Refactor floodsub.publish 
Passed the first test of floodsub
 2019-07-25 14:08:16 +08:00
mhchia
93cf5a2c32
A roughly skeleton of floodsub.publish 
Still need to ensure when to deliver to ourselves
 2019-07-24 22:33:32 +08:00
mhchia
3f52b0dc0a
Remove leftover imports 2019-07-24 21:57:46 +08:00
mhchia
b528c211b9
Temp modified publish 2019-07-24 21:55:57 +08:00
mhchia
218bdb42c4
Add basic functionalities of publish 2019-07-24 21:55:04 +08:00
mhchia
8f9e5a28ff
Temp 2019-07-24 21:54:36 +08:00
Kevin Mai-Husan Chia
f329c5a627
Merge pull request #188 from mhchia/feature/add-typing-for-pubusb 
Add tox and mypy
 2019-07-24 21:45:19 +08:00
mhchia
d3a948be47
Fix error: Change params floodsub.publish back 2019-07-24 16:24:14 +08:00
mhchia
d6c19e71a6
Add typing and notes in pubsub 2019-07-24 14:54:30 +08:00
NIC619
99252e49f8
Prevent re-adding peers to mesh 2019-07-23 23:00:43 +08:00
NIC619
e52bfe3a51
Fix: 
Respond GRAFT with PRUNE if not subscribed to the topic
 2019-07-23 22:52:24 +08:00
NIC619
afc853a776
Apply PR feedback 2019-07-22 23:22:07 +08:00
NIC619
67f9edb77d
Remove fanout topic after joining the topic 2019-07-22 19:28:07 +08:00
NIC619
4ab99485a6
Fix lint error 2019-07-21 23:32:54 +08:00
NIC619
41141c028b
FIx: 
check topic exist in `pubsub.peer_topics`
 2019-07-21 23:16:41 +08:00
NIC619
fd1f318b0c
Fix: 
in mesh heartbeat, select from gossipsub peers subscribed to the topic
 2019-07-21 23:14:08 +08:00
NIC619
404dc67e83
Fix: 
prevent selecting peers from topic not in peer topics
 2019-07-21 23:14:08 +08:00
NIC619
b5c3420c16
Refactor gossipsub.join 2019-07-21 23:14:08 +08:00
NIC Lin
1e78c21eca
Merge pull request #181 from NIC619/fix_handle_unsubscribe 
Fix: handle unsubscribe message
 2019-07-21 20:09:51 +08:00
mhchia
8d3b3fdb6c
Add debug msg 2019-07-20 22:47:37 +08:00
Kevin Mai-Husan Chia
085a5e7526
Apply suggestions from code review 
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
 2019-07-20 22:43:32 +08:00
mhchia
73125b99b0
Fix the error due to the breaking change 
In multiaddr, `split` is moved to `Multiaddr`'s method.
 2019-07-20 22:43:31 +08:00
Kevin Mai-Husan Chia
4422888f6f
Merge pull request #180 from mhchia/fix/include-all-pkgs-setuptools 
Change `packages` in `setup`
 2019-07-20 22:39:57 +08:00
NIC619
36575e8c9b
Add check to prevent gossipsub re-join and re-leave 2019-07-19 19:59:48 +08:00
NIC619
89347be526
Prevent self re-unsubscription 2019-07-18 13:26:31 +08:00
NIC619
f25d97fbd3
Prevent self re-subscription 2019-07-15 16:32:05 +08:00
NIC619
6d9ec7a9c5
Handle the unsubscribe case 2019-07-15 16:28:29 +08:00
mhchia
31cd520076
Add the missing __init__.py in pb 2019-07-11 17:50:36 +08:00
mhchia
a7ce230d05
Change params multiaddr to maddr 
To make pylint happy
 2019-07-10 19:33:38 +08:00
mhchia
c8cb375d19
Make pylint happy 
Not sure why it fails now, but happy previously.
 2019-07-10 17:44:14 +08:00
mhchia
fa092505e8
Add missing "package file" 2019-07-10 00:07:58 +08:00
mhchia
10511d4930
Clean up a bit 2019-07-09 23:55:43 +08:00
NIC619
900b6d9f37
Fix docstring 2019-06-21 11:51:11 -06:00
Stuckinaboot
a4529d827d Modify peer_id str 2019-05-22 11:37:07 -04:00
Stuckinaboot
ae44c062df Remove truncating peerid when converting to string 2019-05-22 11:37:07 -04:00
ZX
49384ad3d7
Merge pull request #159 from libp2p/security-dev 
Implement security
 2019-05-07 09:56:19 -04:00
Stuckinaboot
867f3a70f6 Modify stream to be reader_writer 2019-05-07 01:31:54 -04:00
Robert Zajac
9052e8f8bd
The Gossipsub PR (#162) 
* Add handle_rpc call to pubsub

* Scaffold gossipsub functions

* Add timer

* Implement most of mesh construction

* Implement emit and handle

* Implement fanout heartbeat

* Refactor emit

* some gossipsub cleanup and test

* minor lint stuff, more to come

* Implement publish

* Fix comment

* Modify pubsub/gossipsub so that floodsub tests pass using gossipsub router

* Add floodsub tests to gossipsub

* Handle case where select_from_minus, num_to_select > size(pool-minus)

* Add topic membership

* Implement handle ihave

* Implement most of iwant

* Add mcache.add and comments

* Refactor handle_ihave

* Implement stream write in handle_iwant

* Implement gossip heartbeat

* unresolved vars

* initial mcache code

* documenting mcache

* writing test/debugging mcache

* finished mcache test and debugged

* Make gossipsub backward compatibility its own file

* remove mcache prints

* DEBUGGING

* Add sender_peer_id to handle_rpc to get gossip test passing

* Modify gossipsub to make fanout work

* fanout maintenance test

* debugging gsub GOSSIP

* DEBUGGING

* debugged sender seen cachce

* adding lru, removing prints

* pylint cleanup

* Fix github comments in PR

* minor floodsub possible bugfix
 2019-05-06 23:44:13 -04:00
Alex Haynes
eea6a9fda7
Merge pull request #157 from libp2p/router-refactor 
Refactored RoutedHost into Injected Router
 2019-05-05 14:45:22 -04:00
Alex Haynes
ea4fd8e81f requested changes 2019-05-05 14:32:41 -04:00
stuckinaboot
ff500e6d8e
Merge branch 'master' into security-dev 2019-05-01 17:26:23 -04:00
Stuckinaboot
144dc8d854 Move simple security to libp2p/security 2019-05-01 17:21:11 -04:00
Stuckinaboot
4333c2d061 Fix linting issues' 2019-05-01 13:54:19 -04:00
Alex Haynes
0f5854a83b
Merge pull request #160 from libp2p/providers 
implement add get providers
 2019-04-30 21:48:17 -04:00
Stuckinaboot
e555f17a7b Fix bug in security multistream 2019-04-30 16:07:26 -04:00
Stuckinaboot
f59f27d4d0 Integrate security selectin into libp2p system 2019-04-30 03:09:05 -04:00
Stuckinaboot
1fb4372ede Refine security 2019-04-29 18:05:38 -04:00
zixuanzh
e1d6fdae73 pass test 2019-04-28 18:18:56 -04:00
Alex Haynes
2efc0d3b5c updated peerinfo encoding stored in DHT 2019-04-28 17:38:40 -04:00
zixuanzh
8ab387ac05 refactor add_router in swarm 2019-04-28 15:00:43 -04:00
ZX
21a99287b1
Merge branch 'master' into router-refactor 2019-04-28 14:51:54 -04:00
zixuanzh
3b1a3fb198 fix tests 2019-04-28 13:58:57 -04:00
Alex Haynes
fdaa4f56b9 working on adding tests 2019-04-27 21:59:25 -04:00
Stuckinaboot
eb26661652 Remove outdated encryption folder 2019-04-27 20:09:25 -04:00
Stuckinaboot
d861a00d60 Scaffold security 2019-04-27 19:42:05 -04:00
zixuanzh
4436886371 implement add get providers 2019-04-25 13:25:09 -04:00
zixuanzh
35f75c4dcd move transport to swarm constructor 2019-04-24 22:36:09 -04:00
Alex Haynes
159301306d Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:26:17 -04:00
Alex Haynes
7962060cc0 refactored routedhost into router passed to swarm 2019-04-24 22:17:35 -04:00
Alex Haynes
6c5bac53d7 refactored routedhost into router passed to swarm 2019-04-24 22:11:54 -04:00
Alex Haynes
c4f8bf6ce1 fixed requested changes 2019-04-24 20:59:22 -04:00
Alex Haynes
73ce3461e2 fixed typo 2019-04-21 13:45:18 -04:00
Alex Haynes
9c5cd05042 updated id access in kadpeerinfo 2019-04-21 13:27:44 -04:00
Alex Haynes
5fc57c1076 removed duplicate code 2019-04-21 13:22:11 -04:00
Alex Haynes
e7424d3673 added RoutedHost and updated new_node to support it 2019-04-20 17:35:05 -04:00
Alex Haynes
b0edc47b9a removed commented out code 2019-04-19 21:04:18 -04:00
Alex Haynes
dbe3fcbf5a updated same_home_as to work for variable number of addresses 2019-04-19 21:00:55 -04:00
zixuanzh
69f9aa5f0e refactor KadPeerInfo construction 2019-04-19 20:44:17 -04:00
zixuanzh
1512ae27a1 fix all tests 2019-04-19 20:09:32 -04:00
zixuanzh
fb687dad09 refactor peerinfo 2019-04-19 20:09:32 -04:00
zixuanzh
c03f2f63d2 replace id with peer_id 2019-04-19 20:09:32 -04:00
zixuanzh
9ddbd18ded replace node with KadPeerInfo 
replace node with kadpeerinfo
 2019-04-19 20:09:18 -04:00
ZX
14da7d49fe
Merge branch 'master' into peer_routing 2019-04-17 21:22:37 -04:00
Alex Haynes
d5c7cc7cb7 added changes from future commits to last passing bmuller commit 2019-04-17 20:21:59 -04:00
zixuanzh
d04798ce7c lru cache seen_messages 2019-04-05 21:46:03 -04:00
Stuckinaboot
686c55b09c Remove unnecessary print #146 2019-04-05 17:38:29 -04:00
Stuckinaboot
feaa393c5f Fix seen messages bug 2019-04-03 23:32:03 -04:00