Commit Graph

123 Commits

Author SHA1 Message Date
NIC619
76af835af8
Handle MultiselectError in stream_muxer.accept_stream 2019-09-15 17:35:01 +08:00
NIC619
879f193aa1
Handle errors from
- `read_delim`
    - `read_varint_prefixed_bytes`
    - `decode_uvarint_from_stream`
2019-09-15 16:58:08 +08:00
mhchia
31fb4e0b69
Rewrite _wait_for_data, to handle task precisely
Make the futures first, and then we can compare them with the return
value from `asyncio.wait`.
2019-09-10 23:38:45 +08:00
mhchia
df87f5adb9
Add tests against the daemon for close/reset 2019-09-10 18:01:16 +08:00
mhchia
e5eb01d22b
Fix stream read 2019-09-10 18:01:16 +08:00
mhchia
df312f3e57
Fix linting 2019-09-10 18:01:15 +08:00
mhchia
be2c0f122a
Fix close behavior 2019-09-10 18:01:14 +08:00
mhchia
6c1f77dc1a
Fix: Change the event.close to event.set
And add missing parts.
2019-09-06 21:35:15 +08:00
mhchia
649a230776
Fix MplexStream.read 2019-09-06 17:26:40 +08:00
mhchia
95926b7376
Temp for mplex_stream 2019-09-06 01:08:42 +08:00
mhchia
207fa75d8f
Add reset and close 2019-09-05 23:44:22 +08:00
mhchia
10415cb956
Use ReadWriteCloser for conns and streams 2019-09-05 23:24:17 +08:00
mhchia
eac159c527
Restructure mplex and mplex_stream 2019-09-05 22:29:33 +08:00
mhchia
96230758e4
Add events in MplexStream
And modify a little bit of `close` and `reset`
2019-09-05 18:18:39 +08:00
mhchia
c61a06706a
Refactor interop tests and factories
- Add `close` and `disconnect` in `Host`
- Add `close` and `close_peer` in `Network`
- Change `IListener.close` to async, to await for server's closing
- Add factories for security transports, and modify `HostFactory`
2019-08-29 21:38:06 +08:00
mhchia
34a4d7b0ed
Add the missing StreamID class 2019-08-28 21:45:18 +08:00
mhchia
d35b8ffc64
Conform stream_id to go-mplex 2019-08-28 21:43:34 +08:00
Alex Stokes
a193ae81fd
Fix typo w/ header tag under different reset scenarios 2019-08-26 11:39:30 -07:00
mhchia
b6c8ab0dc9
Fix #259: Use the unsigned LEB128 impl from py-wasm
Reference: https://github.com/ethereum/py-wasm/blob/master/wasm/parsers/leb128.py
2019-08-26 20:41:10 +08:00
mhchia
d59870ebbf
Fix MplexStream error
When receiving a `NewStream`, the message of that packet is the
name of the stream, which should be handled, rather than letting it go
into the message queue.
2019-08-26 20:39:08 +08:00
NIC619
5e68aff1d1
Fix conn attribute and docstring in mplex 2019-08-25 14:42:44 +08:00
Alex Stokes
5b32bfdd3f
Add call to wait_closed method of asyncio.StreamWriter 2019-08-24 22:06:24 +02:00
Alex Stokes
9c5fb4fa5a
Encapsulate concept of a "stream id" to a "muxed" connection 2019-08-24 21:50:07 +02:00
Alex Stokes
e29c1507bf
remove unused fields 2019-08-24 21:50:06 +02:00
mhchia
ef476e555b
Use RawConnection.read
Instead of accessing its reader and writer directly.

TODO: considering add `ReaderWriterCloser` interface and let connection
and stream inherit from it.
2019-08-22 22:53:48 +08:00
mhchia
0b466ddc86
Add lock to RawConnection
To avoid `self.writer.drain()` is called in parallel.
Reference: https://bugs.python.org/issue29930
2019-08-22 22:53:47 +08:00
mhchia
22b1a5395d
A working plaintext 2.0 without validation 2019-08-22 22:53:46 +08:00
mhchia
a0923d202a
Move varint and delim read/write to toplevel
To `libp2p.utils`.
2019-08-22 22:53:46 +08:00
mhchia
7bc363f2fa
Remove initiator in Mplex
Besides, fix the wrong passed `multi_addr` to `mplex_stream`.
2019-08-22 22:53:45 +08:00
mhchia
550c23f9f9
PR feedback
- Use the order in `MuxerMultistream` as the precedence in multiselect
2019-08-21 11:43:25 +08:00
mhchia
8596f7390f
PR feedback: set protocol_id to constants 2019-08-21 11:43:24 +08:00
mhchia
d7d8440b2c
PR feedback: nitpicks 2019-08-21 11:43:24 +08:00
mhchia
4358a4bc89
Negotiate multiselect version for Muxer
`MuxerMultistream` is introduced to negotiate `Multiselect` version
before negotiating Multiplexer's version. This is required by
multistream 1.x
2019-08-21 11:43:24 +08:00
mhchia
86d4ce1da8 Add delim_encode and delim_read
- Add `StreamCommunicator` and `RawConnectionCommunicator`, read/write
messages with delim codec, with `IMuxedStream` and `IRawConnection`
respectively.
- Use it in `Multiselect` and `MultiselectClient`.
2019-08-18 19:51:04 +08:00
mhchia
c5f32bf431 PR feedback for MplexStream.read 2019-08-16 11:03:16 +08:00
Kevin Mai-Husan Chia
8699568d43 Update libp2p/stream_muxer/mplex/mplex_stream.py
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
2019-08-16 11:03:16 +08:00
mhchia
b27cd0f24f Use bytearray over bytes
To avoid copies.
2019-08-16 11:03:16 +08:00
mhchia
92320523d5 Add the missing exceptions.py 2019-08-16 11:03:16 +08:00
mhchia
e37b8bcf19 mypy: Add read_buffer_nonblocking in Mplex 2019-08-16 11:03:16 +08:00
mhchia
f281e3e1db flake8 2019-08-16 11:03:16 +08:00
mhchia
9f8276fa84 Support read(n=-1)
Now, `n=-1` indicates that we want to read until EOF. However, now we
only read until we have no new message.
2019-08-16 11:03:16 +08:00
mhchia
9cb6ec1c48 Modify the behavior of MplexStream.read 2019-08-16 11:03:16 +08:00
mhchia
2485a00e24 Modify NetStream to read n bytes 2019-08-16 11:03:16 +08:00
Alex Stokes
82bae341a7
Run isort over files that were missing it 2019-08-15 16:33:35 -07:00
Alex Stokes
0ebc8ffb21
Wire some missing properties up 2019-08-15 16:33:34 -07:00
Alex Stokes
1e5357a1e1
Update the ISecureConn interface following the reference and simplify accordingly 2019-08-15 16:33:33 -07:00
Alex Stokes
3debd2c808
Run black and isort w/ the new config 2019-08-13 14:36:42 -07:00
Chih Cheng Liang
28f6de37ee
Fix the rest of the typing hints (#232)
* ignore kad

* fix swarm, and minor

* fix init and swarm

* ignore pb

* enable mypy

* fix basic host

* fix tcp

* fix mplex

* add typing for pb

* skip format pyi

* [mypy] no need to ignore pb now

* add typing to chat
2019-08-11 16:47:54 +08:00
Chih Cheng Liang
cb3a59e0ac
ttl as int 2019-08-05 17:02:18 +08:00
Chih Cheng Liang
e763f57930
run isort 2019-08-05 11:47:23 +08:00
Chih Cheng Liang
7a04ebb51f
run black 2019-08-05 11:21:20 +08:00
Chih Cheng Liang
9e0a806218
move stream and connection interfaces to abc 2019-08-05 11:17:38 +08:00
Chih Cheng Liang
29091266fc
add still needed TYPE_CHECK 2019-08-05 10:46:49 +08:00
Chih Cheng Liang
87ef2e4618
remove if TYPE_CHECKING as much as possible 2019-08-05 10:46:49 +08:00
Chih Cheng Liang
4c9a930f84
stream_muxer done 2019-08-05 10:45:47 +08:00
Chih Cheng Liang
dadac423f2
typed muxed_connection_interface.py 2019-08-05 10:45:47 +08:00
Chih Cheng Liang
b64ed9fd6f
typed mplex.utils 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
239a5c88fb
add typing to mplex 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
36b7e8ded9
Refactor HeaderTags 2019-08-05 10:45:46 +08:00
Alex Stokes
7477b29508 run black w/ extended line length 2019-08-04 12:37:41 +08:00
Alex Stokes
c8005c8113 Run isort in repo 2019-08-03 17:50:14 +08:00
Alex Stokes
28e1a03dc4
remove empty yamux impl 2019-08-02 15:03:59 -07:00
mhchia
2e94fcf56c
Remove pylint:disable 2019-08-02 23:26:06 +08:00
Alex Stokes
0ae9840928
Run black over repo 2019-07-31 15:00:12 -07:00
NIC619
5e215901c0
Apply PR feedback 2019-07-30 23:41:28 +08:00
NIC619
c4105688d1
Fix after rebase 2019-07-30 17:31:08 +08:00
NIC619
e53727d301
Apply PR feedback: fix type hints 2019-07-30 16:28:25 +08:00
NIC619
d716e90e17
Fix on type hints 2019-07-30 16:25:33 +08:00
Stuckinaboot
f59f27d4d0 Integrate security selectin into libp2p system 2019-04-30 03:09:05 -04:00
stuckinaboot
57077cd3b4 [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-03-23 13:52:02 -04:00
Robert Zajac
82840b5e6c
Stream rearchitecture (#126)
* Add generic protocol handler

* Add generic protocol handler to stream muxing pipeline

* Modify conn_handler to only deal with connections

* mplex accept stream architecture changes

* Add create generic protocol handler

* Fix minor bugs

* who would win 4 devs or one not

* Debugging

* rearch with handle_incoming infinite loop, seems to work, needs cleanup"

* passing linting, still needs cleanup

* fixing linting again; code still needs cleanup

* fixing tests; code still needs cleanup

* adding test cleanup and task cleanup, removing prints

* linting, and cleanup complete

* storing connections based on peer id

* remove dead code

* remove unnecessary peer_id
2019-02-24 20:58:23 -05:00
Alex Haynes
7736d2afd2
updated reading to read until you see a message for your stream (#100)
* updated reading to read until you see a message for your stream

* added timeout to decode uvarint

* resolved comments

* shortened long line
2019-01-28 16:15:22 -05:00
Alexander Koshkin
ce6ddb27a9 Migrate to new project structure. 2019-01-09 21:38:56 +03:00