Compare commits

...

1281 Commits

Author SHA1 Message Date
Kevin Mai-Husan Chia d2c2a5f933
Merge pull request #431 from wolfgang/patch-1
Don't hash data passed to nacl library in Ed25519PublicKey.verify
2021-03-17 15:45:46 +08:00
Wolfgang Deutsch 0a9f8d23a3
Update ed25519.py
Don't hash data passed to nacl library in Ed25519PublicKey.verify
2021-03-16 08:55:44 +01:00
Kevin Mai-Husan Chia 2efc072ad9
Merge pull request #425 from MohamedAlFahim/patch-1
Fix broken release notes link
2021-02-25 13:13:59 +08:00
Kevin Mai-Husan Chia 55e6ed6e6c
Merge pull request #426 from McSinyx/trio-lowlevel
Use trio.lowlevel instead of trio.hazmat
2021-02-25 13:11:35 +08:00
Nguyễn Gia Phong 080f8edc8e Use trio.lowlevel instead of trio.hazmat
Since trio 0.15.0, hazmat has been deprecated.

trio-typing and mypy are bumped to support newer trio and each other.
2021-02-23 22:02:34 +07:00
Piper Merriam 12786f4e26
Merge pull request #424 from g-r-a-n-t/master
Interop tests updated and fixed.
2020-09-01 19:51:21 -06:00
Mohamed Al-Fahim 69ffaa08d7
Fix broken release notes link 2020-08-27 01:01:45 +00:00
Grant Wuerker 36a4a9150d
Interop tests updated and fixed. 2020-08-25 15:59:22 -06:00
Alex Stokes 5144ab8289
Merge pull request #411 from libp2p/release-notes-fix
Release notes fix: v0.1.15 -> v0.1.5
2020-03-25 16:35:14 -07:00
Jason Carver e712995024 Release notes fix: v0.1.15 -> v0.1.5
Also, update to the actual release date.
2020-03-25 15:59:45 -07:00
Jason Carver 612a2f7c51
Bump version: 0.1.4 → 0.1.5 2020-03-25 15:50:57 -07:00
Alex Stokes c5e1d3263f
add release notes for v0.1.15 2020-03-23 18:06:40 -07:00
Alex Stokes 379a157d6b
Merge pull request #410 from ralexstokes/configurable-msg-id
Allow `Pubsub` creator to supply a custom msg_id
2020-02-28 08:42:49 -08:00
Alex Stokes 857cc29d27
add content-addressed message id utility 2020-02-28 08:25:03 -08:00
Alex Stokes ef666267bd
Allow `Pubsub` creator to supply a custom msg_id 2020-02-28 08:25:03 -08:00
Kevin Mai-Husan Chia 9d68de8c21
Merge pull request #406 from mhchia/feature/noise-patterxx
Noise: `PatternXX` and testing with go
2020-02-28 17:13:00 +08:00
mhchia ec8c10965d
travis CI: use the latest go version 2020-02-28 15:50:03 +08:00
mhchia 1af12ddf43
interop test: support both noise and plaintext 2020-02-28 15:50:03 +08:00
mhchia cb4e1115c6
Remove wrong encoding 2020-02-28 15:50:03 +08:00
mhchia c1f6054b3c
plaintext: update pb 2020-02-28 15:50:03 +08:00
mhchia 728d9e5c9c
Noise: update pb 2020-02-28 15:50:02 +08:00
mhchia f8240bd2cb
plaintext: use varint msg read/writer 2020-02-28 15:50:02 +08:00
mhchia 6016ea731b
`BaseMsgReadWriter`
- Change `BaseMsgReadWriter` to encode/decode messages with abstract
method, which can be implemented by the subclasses. This allows us to
create subclasses `FixedSizeLenMsgReadWriter` and
`VarIntLenMsgReadWriter`.
2020-02-28 15:50:01 +08:00
mhchia 88f660a9c5
Noise: try to use noise in go 2020-02-28 15:50:01 +08:00
mhchia a9f4f285ee
Noise: clean up dup code 2020-02-28 15:50:01 +08:00
mhchia 13e8f496a7
Noise: add noise option in the factories and tests 2020-02-28 15:50:01 +08:00
mhchia 1d2a976597
Remove TODOs 2020-02-28 15:50:00 +08:00
mhchia 48a7c1a969
Security: ensure remote pubkeys are stored 2020-02-28 15:50:00 +08:00
mhchia e02bba93b1
Noise: fix docs
`connection.py` is removed.
2020-02-28 15:49:59 +08:00
mhchia f0df2d189f
Plaintext: use existing msgio reader 2020-02-28 15:49:59 +08:00
mhchia 3c2e835725
Security: `SecureSession`
Make security sessions(secio, noise) share the same implementation
`BaseSession` to avoid duplicate implementation of buffered read.
2020-02-28 15:49:59 +08:00
mhchia 2df47a943c
Refactor 2020-02-28 15:49:59 +08:00
mhchia 874c6bbca4
Refactor MsgIOReadWriter
- Abstract it as `MsgReadWriter`
- `MsgIOReadWriter` as a subclass of `MsgReadWriter`
2020-02-28 15:49:58 +08:00
mhchia ea645f0bd6
Copied read/write from secio 2020-02-28 15:49:58 +08:00
mhchia f8739268e2
Noise: encrypt and decrypt in `NoiseConnection`
TODO: Add a buffer to read only `n` bytes in `read(n)`
2020-02-28 15:49:58 +08:00
mhchia 6ea96e9313
Add comments in pattern xx 2020-02-28 15:49:57 +08:00
mhchia d0290d2b5a
Noise: complete handshake process
TODO
- Figure out why `state.rs` is erased at some moment(even handshake
is not done).
- Refactor
- Add tests
2020-02-28 15:49:56 +08:00
mhchia 8a4ebd4cbb
Support read/write of noise msg and payload 2020-02-28 15:49:56 +08:00
Kevin Mai-Husan Chia 1f881e0464
Merge pull request #405 from mhchia/feature/noise
Noise: skeleton of transport and connection
2020-02-28 15:44:49 +08:00
mhchia 4620544d45
Noise: fix docs 2020-02-15 12:35:36 +08:00
mhchia f4c545ed68
isort: add `noise` to 3rd party config 2020-02-15 12:18:44 +08:00
mhchia 95959725db
Noise pattern: Fix flake8 2020-02-15 12:18:19 +08:00
mhchia d7fabab3e1
Noise: add compiled pb2.py 2020-02-09 13:23:12 +08:00
mhchia 0324a69841
Noise: add `PatternXX` 2020-02-09 00:33:26 +08:00
mhchia f27db83a14
Noise: add TODO comments 2020-02-08 10:48:29 +08:00
mhchia fb53edbc04
Change `async def write`
To return `None` instead of `int. `Writer.write` *does* write all data
in all use case.
2020-02-08 10:24:37 +08:00
Kevin Mai-Husan Chia 1152f9b703
Update libp2p/security/noise/transport.py
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
2020-02-08 10:14:35 +08:00
mhchia 23ece34157
test_noise_connection: fix missing nursery 2020-02-07 18:17:15 +08:00
Kevin Mai-Husan Chia 99f505d6d7
Merge pull request #403 from hukkinj1/disable-incremental-mypy
Disable incremental mode of mypy
2020-02-07 18:10:34 +08:00
mhchia 897e66b7e1
Add the skeletons of noise transport and conn 2020-02-07 17:47:50 +08:00
Taneli Hukkinen 2f3c5b2ee0 Disable incremental mode of mypy 2020-02-06 13:21:36 +01:00
Kevin Mai-Husan Chia e63584c387
Merge pull request #404 from libp2p/feature/trio
Merge `feature/trio` into `master`
2020-02-06 10:49:53 +08:00
mhchia f1dbd52d67
Merge branch 'master' into feature/trio 2020-02-06 10:39:54 +08:00
Kevin Mai-Husan Chia 7c74e36d41
Merge pull request #365 from mhchia/feature/porting-to-trio
Porting to trio
2020-02-06 10:14:11 +08:00
mhchia ddbedc6c15
Pubsub: `handle_talk`
- Change from async function to sync
- Change the name to `notify_subscriptions`, which is clearer.
2020-02-05 21:44:33 +08:00
mhchia 5b03a7ad9f
Mplex: only close the send of new stream channel 2020-02-05 21:41:28 +08:00
mhchia b7c2ec2187
Mplex: change the reference url
To the commit hash, to make it more correct.
2020-02-05 21:31:04 +08:00
mhchia 7f8c0f11f6
Pubsub: change channel size
To `32` to conform to the go implementation.
2020-02-05 21:30:26 +08:00
mhchia ba0fb8a833
Fix: use `pass` over `...`(Ellipse)
Use `...`(Ellipse) only in abstract methods.
2020-02-05 20:36:42 +08:00
mhchia 1fff6ad6b4
Mplex: change message channel size to 8
To avoid infinity sized channel, and to conform to the go
implementation.
2020-02-05 20:31:18 +08:00
mhchia 64c9c48dac
Mplex: change new stream channel size
To `0`, i.e. no unbuffered, to avoid growing buffer size.
2020-02-05 19:48:02 +08:00
mhchia 1e7d5c73ee
test_mplex_stream: refactor 2020-02-05 17:25:39 +08:00
mhchia 996b5cf15d
Mplex: catch exceptions from `channel.send` 2020-02-05 17:05:30 +08:00
mhchia 12cb0d9ac4
Swarm: change `notify_xxx` back to async func 2020-02-04 22:56:13 +08:00
mhchia 13930ae718
SwarmConn: perform `close` right away
In `_handle_new_streams`, when the underlying muxed conn is unavailable,
close `SwarmConn` itself right away, to reset all the streams.
Therefore, the stream processed by `_handle_muxed_stream` are conscious
of the fact that they are reset. It allows a more graceful clean up.
2020-02-04 22:51:21 +08:00
mhchia c0ab609559
Mplex: catch `RawConnError` when writing
Also, do nothing in `MplexStream.reset` if `MuxedConnUnavailable` is
raised when sending the message.
2020-02-04 21:57:59 +08:00
mhchia f884bfa39e
SwarmConn: don't access `Swarm.manager`
Open a local nursery instead.
2020-02-04 21:57:11 +08:00
mhchia 0548d28568
Fix: `StreamReset` in the stream handlers
Since we don't catch `Exception` in the stream handlers, catch them in
the stream handlers in the tests.
2020-02-04 20:46:40 +08:00
mhchia a7ba59bf9f
Add a nursery in `Swarm`
To avoid using the one in `Service`
2020-02-04 20:45:58 +08:00
mhchia b007bb4d07
Use the latest async-service 2020-02-04 17:46:30 +08:00
mhchia 89338914d3
Add comment for `serve_tcp` 2020-02-04 17:45:56 +08:00
mhchia 857bb34f4e
Add checkpoints in `PubsubNotifee`
Since some of the methods in `PubsubNotifee` are doing nothing,
add checkpoints to yield control.
2020-02-04 17:45:37 +08:00
mhchia 66975ae3f2
Pubsub: change `run_task` to `run_daemon_task` 2020-02-04 17:43:39 +08:00
mhchia 7ae9de9002
Fix handler in `net_stream_pair_factory`
Change it to async function. It wasn't discovered since we caught all
exceptions raised in stream handlers.
2020-02-04 17:09:26 +08:00
mhchia 3a91f114ab
Swarm: add `default_stream_handler`
Advantage:
- To avoid `None` checks
- If users forget to register a stream handler for `Swarm`,
with the default stream handler, opened streams aren't removed
until `Swarm` finishes.
2020-02-04 17:05:53 +08:00
mhchia 3fc60cb312
SwarmConn: iterate `streams.copy` in `_cleanup`
To avoid `RuntimeError` if `streams` is changed.
2020-02-04 17:04:28 +08:00
mhchia d483982acb
SwarmConn: don't catch exceptions in handler 2020-02-04 15:10:49 +08:00
mhchia 5da102d1c9
Ping protocol: move `with` statement out of `try` 2020-02-04 15:09:42 +08:00
mhchia 113696dce2
TravisCI: use python `3.7` instead of `3.7-dev` 2020-02-03 16:04:32 +08:00
mhchia 22963a3099
Fix trio-typing>=0.3,<0.4
To be consistent with trinity
2020-02-02 18:18:01 +08:00
mhchia 05d5d045ea
Fix pubsub interop: missing unsubscribe_fn 2020-02-02 18:17:22 +08:00
mhchia 1588be2be9
Change the channel size of peer queue
Back to `0`, to avoid unlimited buffer size.
2020-01-31 17:42:47 +08:00
mhchia e57d01f360
PR feedback
- Use f-string
- Fix wrongly indented comments
- Add dep `trio-typing`
2020-01-28 15:48:41 +08:00
mhchia 095a848f30
Add clean-up logics into TrioSubscriptionAPI
Register an `unsubscribe_fn` when initializing the TrioSubscriptionAPI.
`unsubscribe_fn` is called when subscription is unsubscribed.
2020-01-28 00:29:05 +08:00
mhchia c3ba67ea87
Remove locks in `PubsubNotifee`
- Change `open_memory_channel(0)` to `open_memory_channel(math.inf)`, to
avoid `peer_queue.send` and `dead_peer_queue.send` blocking. This allows
us to remove the locks.
- Only catch `trio.BrokenResourceError`, which is caused by Pubsub when
it's closing.
2020-01-27 14:30:44 +08:00
mhchia 92ea35e147
Fix `IPubsub` and add `IPubsub.wait_until_ready` 2020-01-27 00:10:33 +08:00
mhchia e3a1dd62e4
Use new type hinting for trio channel 2020-01-26 23:56:19 +08:00
mhchia 42bc4d5d06
`INetworkService` implement `ServiceAPI` 2020-01-26 23:55:31 +08:00
mhchia ddfbf9ffc8
Use `raise from` to reserve stacktrace 2020-01-26 23:54:29 +08:00
mhchia b85bab1a09
Don't catch `trio.BusyResourceError` 2020-01-26 23:09:56 +08:00
mhchia 5b4b65faa8
Change default value of `read()`
From `n = -1` to `n = None`, to comply with trio API
2020-01-26 23:03:38 +08:00
mhchia 6e01a7da31
PR feedback: async with host.run() 2020-01-26 16:44:42 +08:00
Kevin Mai-Husan Chia 8e0972f256
Merge pull request #389 from ShadowJonathan/f-string_clean
Apply f-string formatting to everything (except logging)
2020-01-20 18:41:20 +08:00
Jonathan de Jong 4c3510f738 Merge branch 'master' into f-string_clean
# Conflicts:
#	libp2p/network/connection/raw_connection.py
2020-01-20 11:25:37 +01:00
Kevin Mai-Husan Chia 0a2d86a3d0
Merge pull request #392 from ShadowJonathan/lru_lint
add lru to known_third_parties (to fix windows lint)
2020-01-20 11:20:12 +08:00
mhchia dcc4aa52fc
Merge branch 'master' into feature/porting-to-trio 2020-01-19 16:56:40 +08:00
mhchia f0c4254bbd
Use `Service` instead of `ServiceAPI`
To fix error with async-service==0.1.0a5
2020-01-18 00:31:39 +08:00
mhchia 6c7aa30191
Add events in Pubsub
To ensure `handle_peer_queue` and `handle_dead_peer_queue` are indeed
run before the tests finish. Previously, we get errors when performing
`iter_dag` after cancellation. This is because `handle_peer_queue` or
`handle_dead_peer_queue` is not actually run before the Service is
cancelled.
2020-01-18 00:17:30 +08:00
mhchia 54871024cc
Pin the version of async-service to a4 2020-01-16 18:54:19 +08:00
mhchia eef241e70e
Make `Mplex` and `SwarmConn` not `Service`
After second thoughts, they seem not a good candidate of `Service`.
The shutdown logic becomes simpler by making them not `Service`.
2020-01-07 21:50:03 +08:00
mhchia eab59482c0
Use the real `get_unused_tcp_port`
To get rid of the fake one
2020-01-07 16:45:59 +08:00
mhchia 45eeb4fba3
Change `notify_xxx` to sync functions
Since we already have `Swarm.run_task`, we can just change notify
functions to sync.
2020-01-07 16:45:06 +08:00
mhchia 4db043a26a
Remove pexpect from tox 2020-01-07 16:23:00 +08:00
mhchia 52f85586b8
Fix docs 2020-01-07 15:41:44 +08:00
mhchia fe4354d377
Fix `tests_interop`
- Remove pexpect
- Use new version of `p2pclient`, which makes use of anyio
- Clean up tests
2020-01-07 14:14:34 +08:00
mhchia 000e777ac7
Try older async-service 2019-12-26 20:44:58 +08:00
mhchia 94f0fcb6ad
Iterate `dead_peer_receive_channel` with async for 2019-12-26 20:44:32 +08:00
mhchia 68c84b273d
Use `cls` over the name of the factory 2019-12-26 20:44:10 +08:00
mhchia 3c98b1973d
Remove useless conftest for pubsub 2019-12-26 20:43:38 +08:00
NIC Lin 4d814f0587
Merge pull request #394 from NIC619/fix_closing_a_reset_connection
Fix `close` an already reset connection
2019-12-26 12:13:26 +08:00
NIC619 3b0386d861
Raise `RawConnError` 2019-12-24 22:27:16 +08:00
NIC619 c62f1f374f
Check if transport is closing before write/close 2019-12-24 22:23:38 +08:00
NIC619 a390d21385
Remove `RawConnError` from exception handling 2019-12-24 21:57:13 +08:00
mhchia fb6076c061
Upgrade to 0.1.0a4
Probably it can solve the dag issue:
https://github.com/ethereum/async-service/issues/12
2019-12-24 21:50:42 +08:00
mhchia 6ae3f5dc1b
Add checkpoints in tests 2019-12-24 21:28:37 +08:00
mhchia 53dbb0aff1
Fix pubsub_notifee.py
For wrong syntax and import
2019-12-24 18:37:59 +08:00
mhchia 573c049d0f
Catch expections in `PubsubNotifee`
Also, add lock to avoid resource race condition
2019-12-24 18:31:39 +08:00
mhchia 2287dc95be
Fix test for `info_from_p2p_addr`
It is because I removed some checks in the function. This checks should
be useless thanks to mypy
2019-12-24 18:08:33 +08:00
mhchia 3372c32432
Fix examples and modify `new_node`
- Fix examples `chat.py` and `echo.py`
    - Use trio directly, instead of `trio-asyncio`
    - Remove redundant code
- Change entry API `new_node` to `new_host_trio`
2019-12-24 18:03:18 +08:00
NIC619 8f52315816
Fix `close` an already reset connection 2019-12-24 16:19:49 +08:00
mhchia 6fe5871d96
Use `async-exit-stack` over contextlib
For `AsyncExitStack`
2019-12-24 14:44:28 +08:00
mhchia ce5663705f
Merge branch 'master' into feature/porting-to-trio 2019-12-24 02:19:43 +08:00
Kevin Mai-Husan Chia c35cb8318a
Merge pull request #393 from ShadowJonathan/hashable_multiaddr
bump multiaddr to 0.0.9
2019-12-23 17:32:08 +08:00
Jonathan de Jong 45ef63af20 bump multiaddr from setup to 0.0.9 2019-12-23 09:24:41 +01:00
Jonathan de Jong adae81ca01 fix logging bug 2019-12-23 09:15:56 +01:00
Jonathan de Jong 96f6b80e51 Merge remote-tracking branch 'origin/master' into f-string_clean
# Conflicts:
#	libp2p/network/swarm.py
2019-12-23 09:14:15 +01:00
Kevin Mai-Husan Chia 9d3312ebaf
Merge pull request #380 from ShadowJonathan/fix_dependencies
Fix dependency issues (and fastecdsa windows depencency)
2019-12-23 14:26:40 +08:00
Kevin Mai-Husan Chia 1571bfac07
Merge pull request #386 from ShadowJonathan/issue_384
fixes #384
2019-12-23 14:25:38 +08:00
Jonathan de Jong a08e749150 add lru to known_third_parties 2019-12-21 12:56:44 +01:00
Jonathan de Jong 17074dded0 add tests to new multiple multiaddr change 2019-12-21 10:35:34 +01:00
Jonathan de Jong 0b97f4d57c remove eth_utils per PR request 2019-12-21 08:59:07 +01:00
Jonathan de Jong df8be6eb09 revert network/tcp.py and add changes to multiaddr_to_socket_fix 2019-12-19 21:09:11 +01:00
Jonathan de Jong 0827d0d9ef add 2 more instances of formatting violations 2019-12-19 18:09:47 +01:00
NIC Lin 28da206aea
Merge pull request #387 from NIC619/fix_inconsistent_pubsub_peer_record_update
Store peer ids in set instead of list and check if peer id exist before access
2019-12-20 00:34:14 +08:00
Jonathan de Jong b1248ff315 enforced f-strings everywhere, %s on logging
extended _multiaddr_from_socket to support UDP and IPv6 automatically
changed TCPListener to use _ip4_or_6_from_multiaddr to get host, and not ip4 only

enforced `from error` everywhere with raises
added call braces to exceptions
2019-12-19 17:31:18 +01:00
NIC619 3c75c85d7f
Fix extra white space 2019-12-19 23:07:20 +08:00
NIC Lin cb80cfc50b
Update libp2p/pubsub/gossipsub.py
Co-Authored-By: Chih Cheng Liang <chihchengliang@gmail.com>
2019-12-19 16:33:56 +08:00
NIC619 74092c1371
Apply PR feedback: update error msg 2019-12-19 16:26:37 +08:00
NIC619 e51d376d5e
Combine `peers_gossipsub` and `peers_floodsub` 2019-12-19 14:44:49 +08:00
NIC619 6cd3eb8fae
Apply PR feedback:
change param type and remove check before `discard`
2019-12-19 14:15:51 +08:00
Jonathan de Jong 1124fc8211 add eth_utils
add fastecdsa-any requirements
2019-12-19 01:26:44 +01:00
Jonathan de Jong 6cf1b98a88 mark explicit modulo formatting (to get started with PR draft) 2019-12-19 00:37:09 +01:00
Jonathan de Jong f54bc9d1af Make linter happy 2019-12-18 19:05:22 +01:00
Jonathan de Jong 81fe4049cf Apply PR review feedback
> add `if not addr` clause back
> use f-strings for exceptions instead of %s
2019-12-18 18:47:03 +01:00
Jonathan de Jong 4e4d91b2e2
Apply PR review suggestion (change "muxed" to "network" in docstrings)
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-12-18 10:54:52 +01:00
NIC619 f3732f9480
Fix tests 2019-12-18 12:37:04 +08:00
Jonathan de Jong 6b75901243 apply PR feedback (remote len == 0 block, remove redundant fixme comment + docstring line)
change wording of final SwarmException to include possible case of no addresses in returned address set

add `from error` in except clause
2019-12-17 20:20:09 +01:00
Alex Stokes 445c0f8e65
Dangling `kademlia` cleanup 2019-12-17 10:11:17 -08:00
Alex Stokes 1355fbae99
Merge pull request #385 from ShadowJonathan/issue_197
fixes #197
2019-12-17 09:35:36 -08:00
NIC619 19ce8a2140
Fix mypy 2019-12-17 21:56:02 +08:00
NIC619 04b9d688f8
Add newsfragment 2019-12-17 19:09:15 +08:00
Jonathan de Jong 3cbe24caab fixes #384
also adds MultiError to libp2p/exceptions.py

and an additional fixme I have noticed
2019-12-17 12:00:11 +01:00
NIC619 474ed41652
Remove dead peer if floodsub write stream fail 2019-12-17 18:48:25 +08:00
NIC619 009df257bc
Check peer id exist in dict before access 2019-12-17 18:47:58 +08:00
mhchia 47d10e186f
Add `SubscriptionAPI`
And `TrioSubscriptionAPI`, to make subscription io-agnostic.
2019-12-17 18:17:28 +08:00
Jonathan de Jong 794d2101e9 fixes #197 2019-12-17 11:00:45 +01:00
NIC619 f10e3099cb
Change type of peers in pubsub from list to set 2019-12-17 17:55:13 +08:00
NIC619 b4bd997932
Fix mypy 2019-12-17 17:49:49 +08:00
NIC619 65766ec9ac
Change type of local peers var from list to set 2019-12-17 17:36:15 +08:00
NIC619 f1d58ef8ff
Change type of peers from list to set:
`peers_gossipsub`, `peers_floodsub` and mesh/fanout peers
2019-12-17 17:30:24 +08:00
NIC619 7d6daa8e10
Minor cleanup:
- remove outdated comment
- add new peer at the end
- turn peers to send from list to set
2019-12-17 17:17:03 +08:00
mhchia fb0519129d
Refine `Mplex.close` and `SwarmConn.close`
Ensure `close` cleans up things and cancel the service finally.
2019-12-17 15:50:55 +08:00
Jason Carver ef31f7f6d6
Bump version: 0.1.3 → 0.1.4 2019-12-12 13:59:52 -08:00
Jason Carver 368ac7ef7f Compile release notes 2019-12-12 13:58:41 -08:00
Jason Carver 23fa86979d Added release notes 2019-12-12 13:53:09 -08:00
Alex Stokes 733b1d08b6
Merge pull request #372 from ralexstokes/add-py36-compatibility
Add py36 compatibility
2019-12-10 17:28:49 -08:00
Alex Stokes 3b9d7c7acd
Apply PR feedback 2019-12-10 17:20:41 -08:00
NIC Lin ad379221b9
Merge pull request #373 from NIC619/refactor_and_cleanup_gossipsub
Refactor and cleanup gossipsub
2019-12-07 16:28:55 +08:00
NIC Lin a675da52ee
Update libp2p/pubsub/gossipsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-12-07 15:46:42 +08:00
mhchia d847e78a83
Add dep `async-service` 2019-12-07 00:19:10 +08:00
mhchia 837a249552
Fix `security` module 2019-12-07 00:14:01 +08:00
NIC619 2d3bfc8184
Apply PR feedback:
use defaultdict and init control message
2019-12-06 23:42:31 +08:00
mhchia 1929f307fb
Fix all modules except for security 2019-12-06 17:06:37 +08:00
Chih Cheng Liang 82dcce214a
Remove kademlia module (#377)
* Remove kademlia and routing/kademlia

* cleanup

* Fix routed_host test

* lint mypy

* fix doc

* remove set_up_nodes_by_transport_and_disc_opt and fix typing
2019-12-06 14:14:33 +08:00
NIC619 db0017ddbb
Fix lint after applying suggestion 2019-12-05 17:33:07 +08:00
NIC Lin b4900d53da
Apply suggestions from code review
Co-Authored-By: Chih Cheng Liang <chihchengliang@gmail.com>
2019-12-05 15:21:09 +08:00
NIC619 67f02c512a
Remove unnecessary check and fix test 2019-12-05 15:10:04 +08:00
NIC619 fae3798ca9
Apply PR feedback: correct the comment in test 2019-12-05 14:40:49 +08:00
NIC619 e6813da5f5
Refactor `_get_peers_to_send` 2019-12-05 14:35:34 +08:00
NIC619 8e591229fd
Update the sleep time in `test_handle_prune` 2019-12-03 23:10:56 +08:00
NIC619 c08b2375e1
Fix: should not remove topic if no peers 2019-12-03 23:10:47 +08:00
NIC619 a9abf1e3dd
Fix list deletion and add list remove check 2019-12-03 22:37:49 +08:00
NIC619 bb15c817b1
Fix var access before assignment 2019-12-03 22:14:45 +08:00
NIC619 ea6cd30a16
Add back some comment and TODO. Add comment to tests 2019-12-03 18:45:33 +08:00
NIC619 60bd4694a4
Extend wait time for test to pass 2019-12-03 18:03:45 +08:00
mhchia e9ab0646e3
Fix Pubsub 2019-12-03 17:27:49 +08:00
NIC619 b405fd76e9
Add test for gossip heartbeat 2019-12-03 15:49:58 +08:00
NIC619 8dec0b111d
Add test for mesh heartbeat 2019-12-03 15:49:45 +08:00
NIC619 5efdf4c703
Group messages for peer in heartbeat 2019-12-03 15:48:23 +08:00
NIC619 ab1500c708
Remove unneccessary check in gossip heartbeat 2019-12-03 15:03:06 +08:00
Alex Stokes d516cf51b8
Add py3.6 to travis config 2019-12-02 16:33:32 -08:00
Alex Stokes a1fd106bf3
Ensure correct names of test envs in circle ci 2019-12-02 16:33:32 -08:00
Alex Stokes 4c0f511516
Add `py36` tox env for testing 2019-12-02 16:33:32 -08:00
Alex Stokes 63fd531ed0
Fixes to add python 3.6 compatibility 2019-12-02 16:33:32 -08:00
NIC619 920cf646ef
Fix lint and add check in fanout heartbeat 2019-12-02 22:49:27 +08:00
NIC619 a7e0c5d737
Add missing cleanup in gossipsub `remove_peer` 2019-12-02 22:41:49 +08:00
NIC619 357341e0d8
Remove unneccessary filter and check in gossipsub 2019-12-02 22:40:35 +08:00
NIC619 c2d88962c7
Add gossipsub `heartbeat_initial_delay` 2019-12-02 16:55:16 +08:00
NIC619 0672f5ae6d
Fix: move heartbeat delay to `heartbeat` 2019-12-02 16:38:48 +08:00
mhchia bdbb7b2394
Add `RoutedHostFactory`
And skip the tests for `RoutedHost` for now, since there are too many to
be fixed in `Kademlia`, and it's not that necessary now.
2019-12-01 19:17:44 +08:00
mhchia eb494e8682
Fix ping protocol 2019-12-01 19:17:32 +08:00
mhchia 6149aacc01
Fix `examples` 2019-12-01 17:55:07 +08:00
mhchia 31bf774a16
Fix tests in `protocol_muxer` and `libp2p` 2019-12-01 17:43:14 +08:00
mhchia 62e47080f5
Fix `tests/network` 2019-12-01 16:51:06 +08:00
mhchia 79fcdf3a02
Update tests in test_tcp.py
Besides, run `make format`
2019-12-01 16:26:16 +08:00
NIC619 50fd0acf41
Cleanup outdated TODOs in gossipsub 2019-11-30 20:19:17 +08:00
NIC619 0a52a05375
Del entry if no more peers subscribe to the topic 2019-11-30 20:02:11 +08:00
NIC619 e59ac6a250
Cleanup TODOs in pubsub 2019-11-30 17:12:37 +08:00
NIC Lin dfdcf524b7
Merge pull request #362 from NIC619/add_signing_and_verification_to_pubsub
Add signing and verification to pubsub
2019-11-30 13:44:12 +08:00
NIC619 658a0ae156
Apply PR feedback:
move signature validation logic into signature validator
2019-11-29 19:37:48 +08:00
mhchia 1e600ea7e0
Fix `Mplex` and `Swarm` 2019-11-29 19:09:56 +08:00
NIC619 1c54c38ca7
Fix lint and add `signing_strict` to interop tests 2019-11-29 17:24:40 +08:00
NIC619 f4e86b1172
Add tests for failed signature validation cases 2019-11-29 14:13:07 +08:00
NIC619 a262b94836
Apply PR feedback:
check if signing key and ID match
2019-11-29 14:12:42 +08:00
NIC619 064c109b64
Fix signature validator:
Add prefix and return verify result
2019-11-28 18:45:00 +08:00
Jason Carver 14bcc2a7a7
Bump version: 0.1.2 → 0.1.3 2019-11-27 16:20:36 -08:00
Jason Carver 3c0675bbba
Fix bumpversion to look for double-quotes 2019-11-27 16:19:24 -08:00
Jason Carver 52ec6a1606 Compile release notes 2019-11-27 16:13:20 -08:00
Jason Carver f28227729c Customize release notes title
towncrier likes to capitalize things, so libp2p was showing up as
Libp2P. Uuuugly.
2019-11-27 16:09:27 -08:00
Jason Carver 40973ea106 Fix "previous" version in bumpversion to: v0.1.2 2019-11-27 16:04:51 -08:00
Jason Carver 7466ace0b8
Merge pull request #370 from libp2p/remove-unbuildable-dependencies-from-docs-extra
Remove unbuildable dependencies from docs extra
2019-11-27 14:37:10 -08:00
Jason Carver ce37082c44 Add release note for #318
Note that these commits are really just a bugfix, but from a release
perspective, it's the first time that docs will be publicly available.
2019-11-27 14:17:13 -08:00
Jason Carver 27ecd4b0ed Mock dependencies that are excluded in readthedocs
It seems preferable to import just fastecdsa. But if you do that,
then some kind of side-effect doesn't happen, which means that `sec1` is
not available as an attribute on `fastecdsa.encoding`.

So we specifically mock the sub-modules as well.
2019-11-27 14:07:14 -08:00
Jason Carver 3e5f883c50 Strip out fastecdsa in setup.py, during doc build 2019-11-27 14:07:14 -08:00
Jason Carver 4f95bdce42
Merge pull request #368 from carver/release-notes
Add release notes for v0.1.3
2019-11-27 10:49:02 -08:00
Jason Carver 57f1f49a0f Set the latest version in release notes 2019-11-27 10:38:08 -08:00
Jason Carver 84b548beae Back-generate release notes for v0.1.3 release 2019-11-27 10:38:08 -08:00
Jason Carver 2a9dc96269
Merge pull request #369 from carver/docs-travis-target
Add docs test to Travis CI
2019-11-27 10:36:15 -08:00
NIC619 d5d6962dce
Update Pubsub fixture and test 2019-11-27 17:15:24 +08:00
NIC619 0fd400fdf8
Sign and verify in Pubsub 2019-11-27 17:15:24 +08:00
NIC619 f3c9972159
Implement Pubsub signature validator 2019-11-27 17:15:24 +08:00
NIC619 683710573e
Add `strict_signing: bool and sign_key` to Pubsub 2019-11-27 17:15:23 +08:00
NIC Lin da10fc8531
Merge pull request #360 from NIC619/store_our_keypair_in_peerstore
Store our keypair in peerstore
2019-11-27 17:14:36 +08:00
Jason Carver abf0da925c Built docs for modules 2019-11-26 17:44:33 -08:00
Jason Carver d503950179 Fix all doc build warnings 2019-11-26 17:44:33 -08:00
Jason Carver 6668e8d339 sphinx to v2, because starting from blank slate 2019-11-26 17:29:13 -08:00
Jason Carver 1564ad659b Convert doc warnings into errors for CI 2019-11-26 17:20:24 -08:00
Jason Carver b672da82ec Fix version parse failure because of double-quote 2019-11-26 17:19:19 -08:00
Jason Carver 69742da4a3 Add docs test to Travis CI 2019-11-26 16:50:48 -08:00
Jason Carver a6864e3bd3
Merge pull request #366 from carver/splice-in-template
Splice in project template
2019-11-26 15:48:17 -08:00
Jason Carver 5d3be4f890 Fix comment about mypy being not semver 2019-11-26 15:38:32 -08:00
Jason Carver 1a283d0b1a Fix merge bug in the readme 2019-11-26 15:23:10 -08:00
Jason Carver f84edfc133 Add missing package test script used by Makefile 2019-11-26 13:33:58 -08:00
Jason Carver d589daf00d Fill in template variables 2019-11-26 13:33:57 -08:00
Jason Carver d1b5a56ccf Match linting rules after merging in template 2019-11-26 13:33:08 -08:00
Jason Carver 27da312285 Splice in project template
I tried to merge it so that future updates from the template will be
much easier.

The template is hosted at:
https://github.com/ethereum/ethereum-python-project-template
2019-11-26 13:30:41 -08:00
NIC Lin cac7e3909a
Merge pull request #357 from NIC619/minor_fix_replace_del
Replace (check and) del pattern with pop method
2019-11-26 19:39:41 +08:00
mhchia ec43c25b45
Rewrite factories, made some of the test running 2019-11-26 19:24:30 +08:00
NIC619 9f3f2c53da
Apply PR feedback:
use defaultdict for peer data map
2019-11-26 16:18:15 +08:00
Chih Cheng Liang 417b5e7d61
remove unused asyncio 2019-11-26 15:27:06 +08:00
Chih Cheng Liang 50db9e1474
add setup.py 2019-11-26 15:27:06 +08:00
Chih Cheng Liang 6ab0e108d3
minor 2019-11-26 15:27:06 +08:00
Chih Cheng Liang a397ccdc04
makes test_mplex_stream.py::test_mplex_stream_read_write work 2019-11-26 15:27:06 +08:00
Chih Cheng Liang c55ea0e5bb
implement trio queue interface 2019-11-26 15:27:05 +08:00
Chih Cheng Liang 41ff884eef
rewrite tcp reader/writer interface 2019-11-26 15:27:05 +08:00
Chih Cheng Liang d4d345c3c7
progressing 2019-11-26 15:27:05 +08:00
Chih Cheng Liang f5c725788e
need manual stop 2019-11-26 15:27:05 +08:00
Chih Cheng Liang ed17bfd663
hack chat example 2019-11-26 15:27:04 +08:00
NIC619 94c7a0bca4
Apply PR feedback and fix 2019-11-26 14:04:28 +08:00
NIC619 fab27b0357
Fix lint 2019-11-26 12:35:50 +08:00
NIC619 ffa73f5649
Check if pubkey matches peer ID before add 2019-11-26 12:33:55 +08:00
NIC619 0bfbdf7fab
Move keypair into `PeerData` 2019-11-26 12:27:59 +08:00
NIC619 828ae69c66
Apply PR feedback: `add_key_pair` 2019-11-26 11:52:31 +08:00
Jason Carver 89363b2d9b Add internal type for release notes 2019-11-25 21:44:29 +01:00
Jason Carver 07eecb5c69 Separate release-note build from release 2019-11-25 21:44:29 +01:00
Christoph Burgdorf 02fe35663c Setup towncrier to generate release notes 2019-11-25 21:44:29 +01:00
Alex Stokes 493224b75e
Merge pull request #359 from ralexstokes/update-protobuf-dep
Update `protobuf` dep to `3.10.0`
2019-11-25 09:53:46 -08:00
Alex Stokes 7850018041
Update `protobuf` dep to `>=3.10.0` 2019-11-25 09:45:30 -08:00
NIC619 076dae50f3
Fix missing `BasicHost` interface update 2019-11-25 22:05:33 +08:00
NIC619 566e4c080d
Fix lint 2019-11-25 17:32:53 +08:00
NIC619 e28a974425
Update peer store initialization in host factory 2019-11-25 17:17:09 +08:00
NIC619 8b4022328d
Update peer store interface 2019-11-25 17:16:47 +08:00
NIC619 e49de15227
Add `get_private_key` to `Host` 2019-11-25 16:57:00 +08:00
NIC619 a63f00d8f8
Store our pubkey/privkey info during
`initialize_default_swarm`
2019-11-25 16:55:55 +08:00
NIC619 144d93a023
Add pubkey/privkey info to `PeerStore` 2019-11-24 18:04:00 +08:00
NIC619 9837f30698
Rename `peer_map` to `peer_data_map` 2019-11-24 16:52:46 +08:00
NIC619 e355cb2600
Apply PR feedback:
Only use pop method if error handling is in place
2019-11-23 16:04:22 +08:00
NIC619 501eef59de
Apply PR feedback:
Only use pop method if graceful failure handling is desired
2019-11-21 14:48:03 +08:00
Chih Cheng Liang bcd7890124
Move test utilities to tools (#356)
* move test factories to libp2p/tools

* remove unused inits

* move pubsub test utils to tools

* cleanup test_interop

* fix typing libp2p/tools/utils

* add typing to pubsub utils

* fix factories typing

* fix typing for floodsub_integration_test_settings

* fix rest of the typing

* fix isort
2019-11-21 11:47:54 +08:00
NIC619 19907e18ec
Replace (check and) del pattern with pop method 2019-11-20 23:06:37 +08:00
NIC Lin 74198c70b1
Merge pull request #355 from NIC619/check_before_del
Check if entry exists in dictionary before delete
2019-11-19 16:23:26 +08:00
NIC619 c0522c1bd9
Check if entry exists in dictionary before delete 2019-11-17 21:52:05 +08:00
NIC Lin 64c49f809f
Merge pull request #351 from NIC619/handle_stream_io_error
Handle stream io error
2019-11-17 15:09:45 +08:00
NIC619 ace5ef69a8
Apply PR feedback:
handle pubsub dead peer when stream closed in gossipsub
2019-11-16 17:03:04 +08:00
NIC619 ccc7879422
Add stream.write error handling in gossipsub 2019-11-16 16:57:11 +08:00
NIC619 7d1f3d6000
Remove outdated comment 2019-11-16 16:56:59 +08:00
NIC619 1cc5a6f58b
Bump pycryptodome version to 3.9.2 2019-11-16 16:56:59 +08:00
NIC619 cbe57cd5d7
Fix lint 2019-11-16 16:56:59 +08:00
NIC619 86e0fa4563
Handle `StreamClosed` in ping protocol handler 2019-11-16 16:56:59 +08:00
NIC619 9be9b4bbfc
Handle `StreamClosed` in pub/gossip/flood-sub 2019-11-16 16:56:59 +08:00
NIC619 c4f9ce6bb3
Handle `StreamClosed` in identify protocol handler 2019-11-16 16:56:58 +08:00
Jason Carver 32aa20ec39
Merge pull request #353 from carver/release-tools
Pre-release testing tools, plus setup.py updates
2019-11-15 12:05:35 -08:00
Jason Carver b8ec43a859 remove type:ignore for working pycryptodome type
I didn't dig deeper, it looks like pycryptodome must have fixed their
pkcs1_15.new andd pkcs1_15.verify type signatures.
2019-11-15 11:25:03 -08:00
Jason Carver cb4b4b8209 setup.py: fix warnings and add description
There were warnings about needing a URL and maintainer.
Also, use the readme as the pypi description.
2019-11-15 11:25:03 -08:00
Jason Carver 444929d5fc Add tool to make package for smoke test 2019-11-15 11:25:03 -08:00
mhchia 56d3e50267
Bump version: 0.1.1 → 0.1.2 2019-11-14 15:29:50 +08:00
NIC Lin a5c3b8dec2
Merge pull request #340 from NIC619/fix_pubsub_stream_to_disconnected_peer
Register for disconnected event notification by pubsub
2019-11-14 15:16:32 +08:00
NIC619 b8c7f0cfff
Fix lint 2019-11-09 23:55:35 +08:00
NIC619 5dfa29a0df
Track tasks created in pubsub and add `close()` 2019-11-09 23:24:09 +08:00
NIC619 93ef36bd86
Clean up peer record if pubsub stream fail 2019-11-09 23:24:09 +08:00
NIC619 eeb87848af
Apply PR feedback:
- fix await stream close/reset
- make `_handle_dead_peer` a sync function
2019-11-09 23:24:09 +08:00
NIC619 d36e323703
Update error handling of pubsub stream handler 2019-11-09 23:24:08 +08:00
NIC619 97b3aca535
Fix:
Force context switch before canceling swarm connection tasks
2019-11-09 23:24:08 +08:00
NIC619 a8d9536b08
Spin up `handle_dead_peer_queue` task 2019-11-09 23:24:08 +08:00
NIC619 84f5210220
Implement `handle_dead_peer_queue` 2019-11-09 23:24:08 +08:00
NIC619 4b15cb1af5
Implement `PubsubNotifee.disconnected` 2019-11-09 23:24:08 +08:00
NIC619 c6c9393f2b
Add `dead_peer_queue` to pubsub 2019-11-09 23:24:08 +08:00
NIC619 3a0c7d06d1
Update comment for `connection.close()` 2019-11-09 23:24:07 +08:00
Alex Stokes 285bb2ed19
Merge pull request #346 from ralexstokes/add-tests-for-identify
Add tests for identify
2019-11-09 01:51:34 +08:00
Alex Stokes 29873584dc
update test for identify and ping using new test fixture via PR feedback 2019-11-07 21:11:35 -08:00
Alex Stokes 30dee28ef2
add asynccontextmanager utility for a pair of connected hosts 2019-11-07 20:59:01 -08:00
Alex Stokes 2a7b43d853
bugfix: return empty bytes immediately if read length is 0 2019-11-07 20:57:55 -08:00
Alex Stokes 4b01c33d54
add some additional logging 2019-11-07 20:57:43 -08:00
Alex Stokes 9c05e2b0bb
add basic test for identify 2019-11-07 08:05:39 -08:00
Alex Stokes 58f360167d
refactor creation of identify response to make testing easier 2019-11-07 08:05:39 -08:00
Alex Stokes 071eccc995
Merge pull request #345 from ralexstokes/install-default-protocols
Install default protocols
2019-11-08 00:05:05 +08:00
Alex Stokes 9a23609b48
type protocol IDs to satisfy mypy 2019-11-07 07:51:44 -08:00
Alex Stokes e61a5a677f
clean up tests w/ default protocols in place 2019-11-07 07:51:44 -08:00
Alex Stokes ed81562a89
Add `identify` and `ping` as default protocols 2019-11-07 07:51:43 -08:00
Alex Stokes cce33b2f50
Merge pull request #337 from ralexstokes/ground-work-for-identify-to-host
Ground work for identify to host
2019-11-07 23:50:36 +08:00
mhchia 10dd997805
Reorganize factories 2019-11-06 12:11:09 -08:00
Alex Stokes 9500bdbf55
Add class attribute for additional property 2019-11-06 12:11:09 -08:00
Alex Stokes a07c3b0fb0
modify factories to fix tests 2019-11-06 12:11:01 -08:00
Alex Stokes 3deccac2da
fix tests 2019-11-06 12:10:58 -08:00
Alex Stokes 0ca3e83540
lintroll 2019-11-06 11:41:28 -08:00
Alex Stokes 693a8cf99a
Default protocols are constructed using a reference to the host 2019-11-06 11:41:28 -08:00
Alex Stokes e0a94b6092
identify handler reads data on request from the host 2019-11-06 11:41:27 -08:00
Alex Stokes 32c55bcaf2
hosts track their public key 2019-11-06 11:41:27 -08:00
Alex Stokes 11db313b17
A mux can provide the protocols it responds to 2019-11-06 11:41:27 -08:00
Alex Stokes 07c09d8d9e
Merge pull request #341 from hukkinj1/master
Add more linting
2019-11-07 04:39:08 +09:00
Alex Stokes d09046f5ca
Merge pull request #343 from ralexstokes/add-pytest-xdist
Add pytest xdist and fix some issues w/ parallelizing tests
2019-11-07 04:34:14 +09:00
Alex Stokes fbdd52cfdc
Go with immutable datatype in lieu of mutable datatype 2019-11-06 11:25:12 -08:00
Alex Stokes 4a80519ba1
Merge pull request #334 from ralexstokes/add-identify-protocol
Adds identify protocol
2019-11-06 11:05:56 +09:00
Alex Stokes c30d9ce397
Fix linter error from merge in #315. 2019-11-05 17:50:43 -08:00
Alex Stokes c8a5f6f8d7
Run `docformatter` over changes 2019-11-05 17:50:11 -08:00
Alex Stokes 17ba328e1a
Add `pytest-xdist` plugin to parallelize tests 2019-11-05 17:50:11 -08:00
Alex Stokes 94984be4df
Avoid hard-coding ports where it is not relevant for Kademlia tests 2019-11-05 17:50:11 -08:00
Alex Stokes 5cb4479534
Modify the ``KademliaServer`` so that lack of port lets the OS choose a free one 2019-11-05 17:50:10 -08:00
Alex Stokes 8ab69b9676
fix medley of typos 2019-11-05 17:37:43 -08:00
Alex Stokes 01dc49164f
fix bug found via typechecker 2019-11-05 17:37:43 -08:00
Alex Stokes 01ebfa760f
Add `identify` protobufs to makefile 2019-11-05 17:37:43 -08:00
Alex Stokes 08b57fa2ee
Add core handler for `identify` protocol 2019-11-05 17:37:42 -08:00
Alex Stokes 700209c50a
Merge pull request #315 from ralexstokes/add-ping-protocol
Add `ping` protocol.
2019-11-06 10:14:39 +09:00
Taneli Hukkinen 23848039fd Use pytest.fail() to fail test 2019-11-05 01:26:08 +01:00
Taneli Hukkinen 69279108bc Add flake8-bugbear 2019-11-04 21:16:09 +01:00
Taneli Hukkinen cab0e0c1c4 Add warn_unreachable=True mypy config 2019-11-04 21:05:12 +01:00
Piper Merriam dcfeb3e38a
Merge pull request #339 from dmuhs/fix/kademlia-docstring
Fix docstring list in KademliaProtocol
2019-10-29 11:31:35 -06:00
Dominik Muhs 817325f341 Fix docstring list in KademliaProtocol 2019-10-29 18:22:13 +01:00
Piper Merriam ded3792924
Merge pull request #331 from dmuhs/fix/docs-format
Add automatic docstring formatting
2019-10-28 09:39:52 -06:00
Dominik Muhs 3439a2c10e Add docformatter to lintroll and tox CI check 2019-10-26 12:45:26 +02:00
Alex Stokes f2bfc68f6d
Re-generate protobufs 2019-10-25 18:51:57 +09:00
Alex Stokes eaa800c356
Merge pull request #332 from dmuhs/refactor/bool-params
Refactor initiator -> is_initiator and other flags/functions
2019-10-25 09:38:52 +08:00
Dominik Muhs bafdd8512d Enforce pre-summary newline in docstrings 2019-10-24 20:10:45 +02:00
Dominik Muhs 87ed98d7af Add newline before docstring summary 2019-10-24 20:10:36 +02:00
Dominik Muhs 09ab6c51ec Fix formatting in secio transport 2019-10-24 19:52:03 +02:00
Dominik Muhs 714126faf5 Update secio tests to new initiator flag 2019-10-24 19:30:57 +02:00
Dominik Muhs b5eeceecbf Rename raw connection initiator flags 2019-10-24 19:28:42 +02:00
Dominik Muhs 5810174374 Rename stream muxer initiator flags 2019-10-24 19:28:19 +02:00
Dominik Muhs c0452c961b Update stream muxer conftest 2019-10-24 19:25:51 +02:00
Dominik Muhs 031b98ddf0 Rename stream muxer initiator flags 2019-10-24 19:25:34 +02:00
Dominik Muhs 4bda366bb8 Rename protocol muxer initiator flags 2019-10-24 19:25:09 +02:00
Dominik Muhs 021c92ea25 Rename secio initiator flags 2019-10-24 19:22:24 +02:00
Dominik Muhs 7ad96d167c Rename InsecureSession initiator flag 2019-10-24 19:18:29 +02:00
Dominik Muhs 18783b82a2 Rename SecurityMultistream initiator flag 2019-10-24 19:17:00 +02:00
Dominik Muhs ee0b4daf1c Rename BaseSession initiator flag 2019-10-24 19:16:08 +02:00
Dominik Muhs afb79da9f8 Fix typo 2019-10-24 19:12:15 +02:00
Dominik Muhs 2e38d5e5fb Rename TransportUpgrader initiator flag 2019-10-24 19:06:49 +02:00
Alex Stokes 477c08da96
Merge pull request #314 from ralexstokes/add-default-protocols
Add default protocols
2019-10-24 20:10:53 +09:00
Alex Stokes d795a9b854
Add tests for `ping` protocol 2019-10-24 19:53:04 +09:00
Alex Stokes 1cf239cce6
Respect a remote close during the `ping` protocol 2019-10-24 19:44:52 +09:00
Alex Stokes e157c3f654
typing fixes 2019-10-24 18:10:56 +09:00
Alex Stokes f24b488f79
handle other side closing their end of the connection during `ping` 2019-10-24 18:10:56 +09:00
Alex Stokes 4a800e5c56
Add first-pass at `ping` protocol 2019-10-24 18:10:55 +09:00
Alex Stokes 1bf0c31d1f
add basic test for default protocols 2019-10-24 17:36:34 +09:00
Alex Stokes 15cabb1c33
Copy default data to avoid undesirable mutations 2019-10-24 17:29:33 +09:00
Alex Stokes d0c8b7d8af
Avoid sharing default dictionary argument across all instances of class 2019-10-24 17:29:16 +09:00
Alex Stokes c92bade815
Add "default protocols" that all hosts have by default 2019-10-24 17:29:16 +09:00
Dominik Muhs 2af97240c5 Fix import formatting 2019-10-24 09:16:58 +02:00
Dominik Muhs 61cd5e5659 Show diff on incorrectly sorted imports 2019-10-24 09:16:49 +02:00
Kevin Mai-Husan Chia 73251a0c36
Merge pull request #328 from NIC619/another_error_handling
Some minor error handlings
2019-10-24 14:53:19 +08:00
Dominik Muhs 3db297fbee Fix mypy errors 2019-10-24 08:53:19 +02:00
Dominik Muhs eef505f2d9 Add automatic docstring formatter and apply 2019-10-24 08:41:10 +02:00
NIC619 d52b093286
Fix mypy error 2019-10-18 15:59:35 +08:00
NIC619 9889cb8ab1
Fix wrong import 2019-10-18 15:44:07 +08:00
NIC619 211e2f6dd0
Catch `PeerDataError` in `PeerStore.get` 2019-10-17 15:52:57 +08:00
NIC619 5063f0e2a6
Fix lint 2019-10-17 15:34:11 +08:00
NIC619 49bd460e37
Catch `SedesException` in `deserialize_public_key` 2019-10-17 15:19:39 +08:00
NIC619 da08d37c38
Add `SedesException` SecioException 2019-10-17 15:03:18 +08:00
NIC619 29bf623d2c
Raise `DecryptionFailedException`
when failed to decrypt read msg
2019-10-17 14:30:30 +08:00
NIC619 5e5c96f1ea
Add `DecryptionFailedException` MsgioException 2019-10-17 14:29:58 +08:00
Kevin Mai-Husan Chia 30aeb35122
Merge pull request #324 from aratz-lasa/issue-280
Implemented Host that includes a routing system.
2019-10-16 17:20:23 +09:00
Aratz M. Lasa ac9feef26c Pull request feedback 2019-10-15 20:32:25 +02:00
Aratz M. Lasa fa1637850e Removed pipenv Pipfiles 2019-10-15 19:27:04 +02:00
Aratz M. Lasa 70ea471491 Blacked 2019-10-15 19:11:11 +02:00
Aratz M. Lasa d1d91e4091 Refactored for 'lint' testenv 2019-10-15 19:02:03 +02:00
aratz-lasa 65b5e7aeea
Update libp2p/peer/peerinfo.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-10-15 16:31:20 +02:00
Aratz M. Lasa 8143563831 Added tests for 'RoutedHost' and modified 'FindPeer' 2019-10-15 01:01:16 +02:00
Aratz M. Lasa 3f24b015ab Implemented Host that includes a routing system. Explicitly separating different Host types as in Go implementation 2019-10-14 00:29:28 +02:00
Kevin Mai-Husan Chia 00f83a3694
Merge pull request #321 from mhchia/fix/version-in-setuppy
Fix: version released mismatches setup.py
2019-10-11 22:46:02 +09:00
mhchia b8963e4433
Fix: version released mismatches setup.py 2019-10-02 16:16:52 +08:00
Kevin Mai-Husan Chia a369a9f605
Merge pull request #323 from mhchia/fix/ignore-typing-for-wait
Ignore typing for `asyncio.wait`
2019-10-02 16:16:01 +08:00
mhchia 82dc5d9e31
Ignore typing for `asyncio.wait` 2019-10-02 15:45:54 +08:00
Kevin Mai-Husan Chia 50a0783b98
Merge pull request #320 from aleph-im/master
Remove forced debug level
2019-10-02 11:36:08 +08:00
Moshe Malawach 986a852e7e Remove forced debug level 2019-10-01 11:17:05 +02:00
Alex Stokes ec015b5a00
Merge pull request #311 from ralexstokes/clean-up-mss-client
Misc cleanups
2019-09-25 11:47:41 -04:00
Alex Stokes ada4d48b6e
remove overly verbose comments 2019-09-24 15:36:50 -07:00
Alex Stokes e6a0361e09
Merge pull request #291 from ralexstokes/add-ed25519-support
Adds support for verifying ed25519 signatures, for secio
2019-09-24 10:21:01 -07:00
Alex Stokes 75ec2facce
linter fix 2019-09-24 10:07:33 -07:00
Alex Stokes 673ce40133
Add basic tests for ed25519 keys 2019-09-24 10:06:35 -07:00
Alex Stokes 487c923791
add ed25519 private key deserializer 2019-09-24 10:06:35 -07:00
Alex Stokes 6f638b7afe
Merge pull request #307 from swedneck/patch-1
Add links to the new bridged chats to README.md
2019-09-24 09:54:47 -07:00
Alex Stokes bbd8279811
Add explicit exception if we are missing a deserializer 2019-09-24 09:51:32 -07:00
Alex Stokes 6e53849604
Delete utils in favor of serialization module 2019-09-24 09:51:31 -07:00
Alex Stokes 5fdca2ffb2
Add public key implementation 2019-09-24 09:51:31 -07:00
Alex Stokes fa7d1d66a8
Fix import path 2019-09-24 09:51:31 -07:00
Alex Stokes b142964d31
Adds support for verifying ed25519 signatures, for secio 2019-09-24 09:51:31 -07:00
Kevin Mai-Husan Chia dbc35e8b9d
Merge pull request #310 from mhchia/fix/missing_await_for_stream_reset_in_pubsub
Add the missing `await` for `stream.reset` in pubsub
2019-09-24 14:04:25 +08:00
Kevin Mai-Husan Chia b53ca5708f
Merge pull request #305 from mhchia/fix/change-notifee-and-add-tests-for-swarm-conn-and-mplex
Change `Notifee`, add tests for `SwarmConn` and `Mplex`
2019-09-24 14:02:58 +08:00
mhchia da34b086d5
Merge branch 'master' into fix/change-notifee-and-add-tests-for-swarm-conn-and-mplex 2019-09-24 13:50:54 +08:00
Kevin Mai-Husan Chia 3e9df896f4
Merge pull request #308 from mhchia/fix/move-deps-on-p2pclient-to-tox
Move interop tests outside `tests`
2019-09-24 13:49:35 +08:00
mhchia b0c919aab7
Add the missing `await` 2019-09-24 13:34:26 +08:00
mhchia e217acf4ac
Add additional sleep
To be more confident that the stream handler is registered in the
daemon.
2019-09-24 13:28:25 +08:00
mhchia 7405f078e6
Raise `read_delim` exception with different msgs
Separate `len(msg_bytes) == 0` and `msg_bytes[-1:] != b"\n"`, to raise
`ParseError` with different messages.
2019-09-24 13:22:25 +08:00
mhchia 37bee9fb16
PR feedback
- Use `TMuxerOptions` and `TSecurityOptions` in libp2p/__init__.py
- Remove the default value for `muxer_transports_by_protocol` in
`MuxerMultistream` and `secure_transports_by_protocol`
`SecureMultistream`
2019-09-24 12:51:59 +08:00
Kevin Mai-Husan Chia 1bd18c84f2
Apply suggestions from code review
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
2019-09-24 12:33:14 +08:00
mhchia 1bfc6b41e4
Add pexpect in isort third party 2019-09-24 12:18:31 +08:00
mhchia d1c25b8b1e
Fix interop pubsub tests and PR feedback
- Use `from_id`, the changed field name in `PSMessage`.
- PR feedbacks
        - Add label `test` in `testenv` in tox.ini, to avoid wrong
dispatching an environment's command in the future.
        - Use `pytest` over `py.test`.
2019-09-24 11:30:52 +08:00
mhchia 19c17dd512
Remove the leftover `test` in testenv commands
To make `tox -e py37-interop` run.
2019-09-23 22:10:47 +08:00
mhchia 006002f687
Move interop tests out of tests
It is moved to the top level package `tests_interop`, to avoid circular
dependency, with the dependency moved to `tox`.
2019-09-23 22:00:40 +08:00
mhchia 95ae718e3d
Raise `ParseError` in `read_delim` 2019-09-23 16:01:22 +08:00
mhchia 92deae41dc
Change `SwarmConn.conn` to `muxed_conn` 2019-09-23 15:46:50 +08:00
mhchia 8d2415a404
Move calls to `Notifee` inside `Swarm` 2019-09-23 15:01:58 +08:00
mhchia 6f8394e4bd
Merge branch 'master' into fix/change-notifee-and-add-tests-for-swarm-conn-and-mplex 2019-09-21 18:34:12 +08:00
Kevin Mai-Husan Chia 4a838033ff
Merge pull request #304 from mhchia/fix/detection-of-close
Detect closed `Mplex`
2019-09-21 18:28:16 +08:00
mhchia 539047be2d
Make `mplex.read_message` handle `RawConnError` 2019-09-21 18:17:00 +08:00
mhchia e44c2145cc
Merge branch 'master' into fix/detection-of-close 2019-09-21 18:05:54 +08:00
Kevin Mai-Husan Chia 99ef3f08d8
Merge pull request #303 from mhchia/fix/refactor-mplex-swarm-host
Refactor: `Mplex`, `Swarm`, and `BasicHost`
2019-09-21 17:27:09 +08:00
mhchia 0df578d219
Add the missing exceptions.py 2019-09-21 17:18:55 +08:00
mhchia a27a817d50
Fix tests 2019-09-20 16:17:13 +08:00
mhchia 89c127eff4
Merge branch 'master' into fix/refactor-mplex-swarm-host 2019-09-20 15:45:28 +08:00
swedneck c2ef724136
change matrix alias 2019-09-20 06:43:49 +02:00
swedneck 96559cb95a
fix typo 2019-09-20 06:40:50 +02:00
swedneck d94a36c42b
Add links to the new bridged chats to README.md
matrix, discord, and IRC are all bridged together via matrix
2019-09-19 19:27:18 +02:00
NIC Lin 85457fa308
Merge pull request #299 from NIC619/add_more_error_handling
Add more error handling
2019-09-19 23:45:02 +08:00
NIC619 7fc958e7be
Add exception raised to docstring 2019-09-19 22:19:36 +08:00
NIC619 c6294ad19b
Raise `MultiselectCommunicatorError`:
when failed to write to communicator
2019-09-19 21:51:23 +08:00
NIC619 b9d1875027
Catch `OpenConnectionError` in `swarm.dial_peer` 2019-09-19 21:24:01 +08:00
NIC619 1f76f6ee1b
Raise `OpenConnectionError`
when failed to open connection
2019-09-19 21:23:35 +08:00
NIC619 c44be5e247
Add `OpenConnectionError` for base transport 2019-09-19 21:23:01 +08:00
mhchia 62b0bc4580
Remove useless protocol_ids in logging 2019-09-19 16:31:42 +08:00
mhchia 313ae45b45
Add tests for `MplexStream` 2019-09-19 16:31:42 +08:00
mhchia 02c55e5d14
Add tests for `MplexConn` 2019-09-19 16:31:42 +08:00
mhchia a9ad37bc6f
Add mplex tests and fix error in `SwarmConn.close` 2019-09-19 16:31:41 +08:00
mhchia d61327f5f9
Add tests for SwarmConn 2019-09-19 16:31:41 +08:00
mhchia b8b5ac5e06
Add test for notifee disconnected 2019-09-19 16:31:41 +08:00
mhchia 675c61ce3b
Move test_notify from libp2p to network 2019-09-19 16:31:41 +08:00
mhchia 5307c0506b
Change `IMuxedConn` to `INetConn` in `Notifee` 2019-09-19 16:31:41 +08:00
mhchia 0356380996
Add tests for swarm, and debug
Fix `swarm_pair_factory`
2019-09-19 16:31:40 +08:00
mhchia 276ac4d8ab
Add initial test for `Swarm.close_peer` 2019-09-19 16:31:13 +08:00
mhchia e7304538da
Add test for `Swarm.close_peer` 2019-09-19 16:31:12 +08:00
mhchia 6923f257f6
Remove print 2019-09-19 16:07:53 +08:00
mhchia 5f064dd329
Refactor: get rid of single huge _handle_incoming 2019-09-19 15:55:27 +08:00
mhchia 4a689c7d57
Fix error when reset
If `Mplex` is cleanup first, `MplexStream.reset` possibly fails because
`Mplex.streams` is set to `None` in `cleanup`.
2019-09-19 15:55:27 +08:00
mhchia b51c2939a8
Handle exceptions inside `read_message`
And remove the need of checking `None` for every read messages.
2019-09-19 15:55:26 +08:00
mhchia f62f07bb9f
Handle `IncompleteRead` in `handle_incoming` 2019-09-19 15:55:26 +08:00
mhchia 2d8e02b7eb
Add detection for disconnections in mplex 2019-09-19 15:55:26 +08:00
mhchia 393b51a744
isort 2019-09-19 15:53:40 +08:00
mhchia 5653b3f604
Add "closed" and "shutting_down" events 2019-09-19 15:53:39 +08:00
mhchia 7cf0495f37
Remove `print` 2019-09-19 15:38:38 +08:00
mhchia a7bc9fc358
Asynchronously handling the accepted stream. 2019-09-19 13:59:51 +08:00
mhchia 0c7afeebaf
Fix test_security_multistream 2019-09-19 13:59:50 +08:00
mhchia 6cb033fd1f
Refactor `multiselect` out of `Swarm` to BasicHost 2019-09-19 13:59:50 +08:00
mhchia 0bd213bbb7
Refactor mplex and start to add close detection 2019-09-19 13:56:05 +08:00
NIC619 7243eb9766
Fix different exception raised in test 2019-09-17 21:44:48 +08:00
NIC619 f253152858
Handle protocol negotiation failure in swarm `new_stream` 2019-09-17 16:17:41 +08:00
NIC619 559f419b4e
Fix stream registration in `accept_stream` 2019-09-17 15:42:18 +08:00
NIC619 0080466d86
Handle `RawConnError` in `InsecureSession.run_handshake` 2019-09-16 19:16:41 +08:00
NIC619 09bfa0ab09
Handle `IOException` in `create_secure_session` 2019-09-16 19:15:40 +08:00
NIC619 c7593bff97
Add `InconsistentNonce` in secio 2019-09-16 19:11:46 +08:00
NIC619 359bcf45ff
`SecioException` inherit from `HandshakeFailure` 2019-09-16 19:11:06 +08:00
NIC619 d6dda91482
Move `HandshakeFailure` to libp2p.security 2019-09-16 19:09:09 +08:00
NIC619 4cd5b77f10
Raise `RawConnError` in `RawConnection` 2019-09-16 18:37:00 +08:00
NIC619 cb632fa509
Add `RawConnError` 2019-09-16 18:35:48 +08:00
NIC619 76af835af8
Handle `MultiselectError` in `stream_muxer.accept_stream` 2019-09-15 17:35:01 +08:00
NIC619 eaa74c4e26
Handle `MultiselectCommunicatorError` 2019-09-15 16:58:22 +08:00
NIC619 879f193aa1
Handle errors from
- `read_delim`
    - `read_varint_prefixed_bytes`
    - `decode_uvarint_from_stream`
2019-09-15 16:58:08 +08:00
NIC619 905a473ac3
Add `MultiselectCommunicatorError` 2019-09-15 16:37:37 +08:00
NIC619 68573e94d3
Have `StreamError` inherit from `IOException` 2019-09-15 16:34:16 +08:00
NIC Lin 7483da762e
Merge pull request #298 from NIC619/add_more_logging
Add a little more loggings
2019-09-15 15:34:28 +08:00
NIC619 f368f5e93b
Apply PR feedback 2019-09-15 15:09:58 +08:00
NIC619 501513b747
Update `IMultiselectClient` 2019-09-14 22:24:53 +08:00
NIC619 9bad7a61f0
Add some loggings to pubsub 2019-09-14 21:54:26 +08:00
NIC619 786a03544c
Add some loggings to swarm and cosmetic updates 2019-09-14 21:47:49 +08:00
NIC619 38f4223e62
Add pip-wheel-metadata to gitignore 2019-09-14 21:47:15 +08:00
NIC619 65a48d5c51
Remove unused `select_protocol_or_fail` 2019-09-14 17:42:18 +08:00
Kevin Mai-Husan Chia 811cd7813a
Merge pull request #292 from mhchia/fix/dial-sig-changed
Fix `transport.dial` in swarm
2019-09-11 18:56:36 +08:00
mhchia 8c8c206c33
flake8 2019-09-11 18:33:52 +08:00
mhchia 451f993058
Fix isort 2019-09-11 18:05:41 +08:00
mhchia c1ffc0ab07
Fix `transport.dial` in swarm 2019-09-11 17:13:21 +08:00
Alex Stokes cb1a25f94c
Execute the todo to remove an unused argument 2019-09-10 21:01:03 -04:00
Alex Stokes 4bd32cc4bc
Add logs during connection handshake 2019-09-10 19:02:29 -04:00
Alex Stokes f128c746f0
Write data payload as hex to log 2019-09-10 16:17:40 -04:00
Alex Stokes 68e75707e4
Enhance logs 2019-09-10 16:04:18 -04:00
Kevin Mai-Husan Chia 988ef8c712
Merge pull request #287 from mhchia/fix/mplex-stream-close-reset
Fix close/reset behavior
2019-09-10 23:57:44 +08:00
mhchia 31fb4e0b69
Rewrite `_wait_for_data`, to handle task precisely
Make the futures first, and then we can compare them with the return
value from `asyncio.wait`.
2019-09-10 23:38:45 +08:00
mhchia df87f5adb9
Add tests against the daemon for close/reset 2019-09-10 18:01:16 +08:00
mhchia bb0da41eda
Remove `cleanup`
`cleanup` cancels all tasks in the loop, including the main one run by
`run_until_complete`
2019-09-10 18:01:16 +08:00
mhchia a45eb76421
Suppress all exceptions in clean up. 2019-09-10 18:01:16 +08:00
mhchia e5eb01d22b
Fix stream read 2019-09-10 18:01:16 +08:00
mhchia df312f3e57
Fix linting 2019-09-10 18:01:15 +08:00
mhchia 0ab548aee5
Add the missing tests 2019-09-10 18:01:15 +08:00
mhchia be2c0f122a
Fix close behavior 2019-09-10 18:01:14 +08:00
Brian Cloutier b2146c5268
Don't crash on large messages 2019-09-09 17:13:26 -04:00
Alex Stokes 155bec0562
Fix initiator flag during secio upgrade 2019-09-09 16:40:14 -04:00
Alex Stokes f9321924ad
Merge pull request #286 from ralexstokes/fix-initiator-flag
Restore `initiator` flag to `BaseSession` type
2019-09-08 16:03:34 -04:00
Alex Stokes 2025a5c7f1
Restore `initiator` flag to `BaseSession` type 2019-09-08 15:40:02 -04:00
Alex Stokes f38899e26e
Merge pull request #284 from ralexstokes/remove-friendly-ids
remove friendly IDs
2019-09-07 11:14:05 -04:00
Alex Stokes 50ae439d20
remove friendly IDs 2019-09-07 11:04:20 -04:00
Alex Stokes 2d1d3d0136
Merge pull request #282 from mhchia/fix/mplex-stream-close-reset
Fix mplex stream close reset
2019-09-06 19:49:03 -04:00
mhchia 6c1f77dc1a
Fix: Change the `event.close` to `event.set`
And add missing parts.
2019-09-06 21:35:15 +08:00
mhchia 1cd969a2d5
Fix: Add typing in functions 2019-09-06 20:02:35 +08:00
mhchia a754e7dbbe
Add the missing tests.constants 2019-09-06 17:59:39 +08:00
mhchia 649a230776
Fix `MplexStream.read` 2019-09-06 17:26:40 +08:00
mhchia 95926b7376
Temp for mplex_stream 2019-09-06 01:08:42 +08:00
mhchia 207fa75d8f
Add `reset` and `close` 2019-09-05 23:44:22 +08:00
mhchia 10415cb956
Use `ReadWriteCloser` for conns and streams 2019-09-05 23:24:17 +08:00
mhchia eac159c527
Restructure mplex and mplex_stream 2019-09-05 22:29:33 +08:00
mhchia 96230758e4
Add events in MplexStream
And modify a little bit of `close` and `reset`
2019-09-05 18:18:39 +08:00
Alex Stokes 34b02588cf
Merge pull request #279 from ralexstokes/echo-demo
Echo demo
2019-09-04 19:54:28 +02:00
Alex Stokes c1fdf9accf
Add echo demo 2019-09-04 10:45:41 -07:00
Alex Stokes 7ed9eac3b6
Merge pull request #278 from ralexstokes/add-buffering-to-secio-reads
Add buffering to secio reads
2019-09-04 19:38:29 +02:00
Alex Stokes 451ec2664a
Address incorrect typing in pycryptodome dependency 2019-09-04 10:19:27 -07:00
Alex Stokes a099b9c65d
Clean up temporary hold over from previous PR 2019-09-04 10:15:30 -07:00
Alex Stokes b214f88f75
Avoid using message-based IO in the `plaintext` protocol
Can reuse the machinery in `secio` but need to generalize the
"buffering" there
2019-09-04 10:12:43 -07:00
Alex Stokes f86ba7283d
Implement signing for RSA
- mainly for use in `secio` w/ RSA-based identities b/t peers
2019-09-04 10:12:43 -07:00
Alex Stokes 25f504ad35
Allow RSA public key type when deserializing keys 2019-09-04 10:12:43 -07:00
Alex Stokes 1a359770dd
Use `msgio` IO and proper buffering in `secio` implementation 2019-09-04 10:12:43 -07:00
Alex Stokes fc3e3a4be5
Changes to type hints to match new abstractions 2019-09-04 10:12:43 -07:00
Alex Stokes 6d97702da7
Merge pull request #277 from ralexstokes/add-io-abstractions
Introduces IO abstractions and classes for `msgio` IO
2019-09-04 19:11:50 +02:00
Alex Stokes a764fd4e6f
simplify some of the msgio class hierarchy via PR feedback 2019-09-04 09:57:04 -07:00
Kevin Mai-Husan Chia e9ca372fb5
Merge pull request #271 from mhchia/fix/pubsub-interop
Pubsub interop with go-libp2p-daemon
2019-09-04 22:28:14 +08:00
mhchia 1f3c9af45b
Add the missing `is_proc_running=True` 2019-09-04 22:19:11 +08:00
mhchia 34b489af25
Fix kad_peerinfo according to peerinfo 2019-09-04 21:37:33 +08:00
mhchia 0e3d4508d6
PR feedback
- Use `Sequence` instead of `List`
- Add note
- Remove redundant words in docstring
2019-09-04 20:52:18 +08:00
mhchia db0da8083a
Do `p2pd.close` if not all of them succeed 2019-09-04 20:52:17 +08:00
Kevin Mai-Husan Chia 51d547ccc5
Update tests/interop/utils.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:38:38 +08:00
Kevin Mai-Husan Chia b72c489f4e
Update tests/interop/daemon.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:36:42 +08:00
Kevin Mai-Husan Chia a843514afb
Update tests/interop/daemon.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:35:42 +08:00
Kevin Mai-Husan Chia 155f523c9f
Update tests/interop/daemon.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:33:50 +08:00
Kevin Mai-Husan Chia bd21b2f66f
Update tests/interop/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:33:29 +08:00
Kevin Mai-Husan Chia 5113785543
Update libp2p/pubsub/pubsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:32:43 +08:00
mhchia 46c13ee1c0
Fix CI to call the script in the correct path 2019-09-04 16:34:56 +08:00
mhchia dddaacad62
Move install script under `tests/interop/go_pkgs` 2019-09-04 16:33:25 +08:00
mhchia 677531db76
Fix pubsub tests 2019-09-04 15:33:07 +08:00
Alex Stokes 2a02f92f77
Temporary porcelain until next PR 2019-09-03 21:59:50 -07:00
Alex Stokes 8e8318aa5c
Introduces IO abstractions apart from `asyncio` or those attached to `IRawConnection`
Also adds `msgio` utilities to mirror the Go implementation
2019-09-03 21:59:50 -07:00
Alex Stokes cd3e093001
remove leftover simple security module 2019-09-03 21:38:33 -07:00
Alex Stokes 506093216c
Merge pull request #276 from ralexstokes/fix-secio-impl
Fix secio impl
2019-09-03 23:01:45 +02:00
Alex Stokes 272ab60d47
Merge pull request #275 from ralexstokes/use-different-ecc-backend
Use a different ECC backend `fastecdsa` with a compatible serializer
2019-09-03 23:00:33 +02:00
Alex Stokes 30456f8018
Use a different ECC backend with a compatible serializer
This library has the ``SEC1`` encoder which is compatible
with the serialization of ECC keys/points used in the Go libp2p impl
2019-09-03 13:44:25 -07:00
Alex Stokes bfd674e22c
Try all pairs of choices, not just a small subset via `zip` 2019-09-03 13:28:31 -07:00
Alex Stokes f8bbaf60a1
Add more information to the peer mismatch exception 2019-09-03 13:28:18 -07:00
Alex Stokes 350cc04c97
Allow the key pair to drive ID generation 2019-09-03 13:26:58 -07:00
Alex Stokes eaeb36c1d9
Merge pull request #274 from ralexstokes/add-identity-hash-for-peer-ids
Add option to inline "short" public keys for peer IDs
2019-09-03 22:21:58 +02:00
Alex Stokes 345e696a7d
Add option to inline "short" public keys for peer IDs
Added to remain interoperable w/ the Go implementation
2019-09-03 13:14:04 -07:00
mhchia 961e51fa2e
Remove leftover prints 2019-09-03 23:39:29 +08:00
mhchia 7f20ab781d
Fix gosssipsub tests 2019-09-03 23:37:34 +08:00
mhchia d7bce941d8
Fix wrong spelling 2019-09-03 23:36:31 +08:00
mhchia b23bf5d704
Avoid isort sorting the import wrong 2019-09-03 23:00:31 +08:00
mhchia 4f7bb30d82
Add `INetStream` to type `StreamReader`
TODO: Make stream readers implement `Reader`
2019-09-03 22:59:44 +08:00
mhchia c6d81d70b3
Try 3.7-dev in CI
Use https over ssh when cloning go-libp2p-daemon

To avoid """
Warning: Permanently added the RSA host key for IP address '140.82.113.4' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
"""

Exit if clone fails

Fix daemon url
2019-09-03 22:50:14 +08:00
mhchia d1b0340164
Update bindings version 2019-09-03 21:55:07 +08:00
mhchia 2ba7948f95
Update bindings version 2019-09-03 18:52:55 +08:00
mhchia 5280f3965c
Update install script for interop
And adjust the structure of go packages for interop
2019-09-03 17:41:17 +08:00
mhchia 749ff275ed
Refactor `make_p2pd`
Let `make_p2pd` get rid of `unused_tcp_port_factory`, which should only
exist in fixtures/tests.
2019-09-03 16:55:42 +08:00
mhchia 7385a7a677
Add `is_gossipsub` fixture in interop test
To use the same code to test against both routers: floodsub and
gossipsub.
2019-09-03 16:49:00 +08:00
mhchia 33dae87c35
Add pubsub test for gossipsub 2019-09-03 16:07:44 +08:00
mhchia fd1f466002
Fix: failed to open stream using existing conn
Fix #233
2019-09-03 14:12:16 +08:00
mhchia 194b494057
Tested against subscriptions and publish 2019-09-02 23:21:57 +08:00
mhchia 3717dc9adf
Add helper functions 2019-09-02 21:01:13 +08:00
mhchia a883816881
Add `connect` utility function 2019-09-02 18:40:12 +08:00
mhchia 56ef0b962c
Add test for `host` `connect` and `disconnect` 2019-09-02 17:32:15 +08:00
mhchia dfd9ebdc5e
Change `PeerInfo` to remove dep on `PeerData` 2019-09-02 14:30:27 +08:00
mhchia b77834d129
Use `asyncio.subprocess` over pexpect
In the test for pubsub, since there were unknown issues when I test
against pexpect.
2019-09-02 14:30:27 +08:00
mhchia 1b5d064a8d
Add utility functions for libp2p bindings
To prepare for pubsub interop test
2019-09-02 14:30:25 +08:00
Kevin Mai-Husan Chia db858e467c
Merge pull request #273 from mhchia/fix/secio-pb-initpy
Add the missing __init__.py
2019-09-02 14:27:59 +08:00
mhchia 77a0cc3a87
Add the missing __init__.py 2019-09-02 14:18:52 +08:00
Kevin Mai-Husan Chia 73ae5a35ec
Merge pull request #270 from mhchia/fix/mplex-stream-id-and-interop
Fix mplex stream id and add interop CI
2019-08-31 23:56:43 +08:00
mhchia b955c0fa02
Explicitly import ID, Transport from secio 2019-08-31 22:38:46 +08:00
mhchia aa0866698f
PR feedback: Add check in `Swarm.close_peer` 2019-08-31 22:37:59 +08:00
Kevin Mai-Husan Chia 9e8a6bdf29
Update tests/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-31 22:32:43 +08:00
Kevin Mai-Husan Chia 1e59438f25
Update libp2p/network/swarm.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-31 22:32:32 +08:00
mhchia cec2aea928
Move shared fixtures and constants to files 2019-08-29 22:38:08 +08:00
mhchia 9ceb5f55bb
Call `make_echo_proc` with `is_host_insecure`
Use the fixture, this way we can configure `is_host_insecure` to support
the test against secio.
2019-08-29 22:08:27 +08:00
mhchia b2c5371323
Add TODO for `Swarm.connections` 2019-08-29 22:00:07 +08:00
mhchia c61a06706a
Refactor interop tests and factories
- Add `close` and `disconnect` in `Host`
- Add `close` and `close_peer` in `Network`
- Change `IListener.close` to async, to await for server's closing
- Add factories for security transports, and modify `HostFactory`
2019-08-29 21:38:06 +08:00
mhchia 64c0dab3af
Fix isort 2019-08-29 00:01:48 +08:00
mhchia e0399beed8
Fix tar argument 2019-08-28 23:50:46 +08:00
mhchia 15f62dff68
Use go12 in CI 2019-08-28 23:48:25 +08:00
mhchia b726d7c9da
Add tox and CI for interop 2019-08-28 23:39:33 +08:00
mhchia 34a4d7b0ed
Add the missing StreamID class 2019-08-28 21:45:18 +08:00
mhchia d35b8ffc64
Conform `stream_id` to go-mplex 2019-08-28 21:43:34 +08:00
Alex Stokes 9b60e1757d
Merge pull request #269 from ralexstokes/add-reset-to-network-stream
Expose `reset` method on `NetStream`
2019-08-26 21:27:57 +02:00
Alex Stokes a193ae81fd
Fix typo w/ header tag under different reset scenarios 2019-08-26 11:39:30 -07:00
Alex Stokes c5eda4065d
Expose `reset` method on `NetStream` 2019-08-26 11:38:39 -07:00
Alex Stokes 98a0e76dda
Merge pull request #268 from mhchia/fix/mplex-interop
Fix: name of a `MplexStream` is not handled in `Mplex`
2019-08-26 19:51:57 +02:00
Alex Stokes 66c3bacf7e
Merge pull request #254 from ralexstokes/implement-secio
Implement `secio`
2019-08-26 19:22:00 +02:00
Alex Stokes fa0acd9fc5
Apply PR feedback 2019-08-26 10:03:12 -07:00
Alex Stokes c1ffb03f77
Update comment to reflect correct function 2019-08-26 09:51:49 -07:00
mhchia b6c8ab0dc9
Fix #259: Use the unsigned LEB128 impl from py-wasm
Reference: https://github.com/ethereum/py-wasm/blob/master/wasm/parsers/leb128.py
2019-08-26 20:41:10 +08:00
mhchia d59870ebbf
Fix `MplexStream` error
When receiving a `NewStream`, the message of that packet is the
name of the stream, which should be handled, rather than letting it go
into the message queue.
2019-08-26 20:39:08 +08:00
NIC Lin 5b122d04b2
Merge pull request #267 from NIC619/fix_conn_attr_in_mplex
Small fix on `conn` attribute and docstring in mplex
2019-08-25 16:53:29 +08:00
NIC619 5e68aff1d1
Fix `conn` attribute and docstring in mplex 2019-08-25 14:42:44 +08:00
Alex Stokes f08aa339b4
Verify the channel can pass some plaintext 2019-08-24 23:26:26 +02:00
Alex Stokes 737195f461
Simplify testing connection w/ other simplifications 2019-08-24 23:15:31 +02:00
Alex Stokes 10e30beb42
Disable "friendly" IDs for tests that expect a full string 2019-08-24 22:57:22 +02:00
Alex Stokes a363ba97d1
Work in terms of the `IRawConnection` abstraction 2019-08-24 22:52:09 +02:00
Alex Stokes eb5ef39399
Convert message IO to work w/ a `RawConnection`. 2019-08-24 22:47:56 +02:00
Alex Stokes 7c004a4e14
Mypy fixes 2019-08-24 22:38:26 +02:00
Alex Stokes 44e5de636f
Add "friendly" peer ID string representation for debugging 2019-08-24 22:38:26 +02:00
Alex Stokes 852609c85d
Clean up base session type 2019-08-24 22:38:26 +02:00
Alex Stokes 9355f33da8
Add basic test for `secio`
Two peers in-memory can create a secure, bidirectional channel
2019-08-24 22:38:26 +02:00
Alex Stokes b8c0ef9ebb
Fix bugs in `secio` implementation 2019-08-24 22:38:25 +02:00
Alex Stokes 228032805a
Some code cleanup 2019-08-24 22:38:25 +02:00
Alex Stokes 3f4589d497
Get tests working 2019-08-24 22:38:25 +02:00
Alex Stokes d176115972
Add `secio` to security upgrader suite 2019-08-24 22:37:46 +02:00
Alex Stokes 376a5d4fc6
Adjust callsite 2019-08-24 22:37:45 +02:00
Alex Stokes 1adef05e94
Typing and linting fixes 2019-08-24 22:37:45 +02:00
Alex Stokes 0fa3331b8c
Add clearer indication of "self encryption" 2019-08-24 22:37:45 +02:00
Alex Stokes 8e913a3faa
Dispatch serialization of keys based on key type
- Add some tests to check high-level roundtrip
2019-08-24 22:37:45 +02:00
Alex Stokes 4d30b31c55
Finish first pass at `secio` implementation 2019-08-24 22:37:45 +02:00
Alex Stokes af2e50aaf4
Add facility for authenticated encryption 2019-08-24 22:37:45 +02:00
Alex Stokes 228c17ae9e
Add ECDH key exchange utility 2019-08-24 22:37:44 +02:00
Alex Stokes fb13dfa7b3
Add `sign` and `verify` operations for `secp256k1` keys 2019-08-24 22:37:44 +02:00
Alex Stokes 3c97a5a0ed
Add ECC key implementation 2019-08-24 22:37:44 +02:00
Alex Stokes 91e11f3ec0
[wip] more secio implementation 2019-08-24 22:37:44 +02:00
Alex Stokes 0cc3fc24a7
Add source for some secure bytes, e.g. to provide a nonce 2019-08-24 22:37:44 +02:00
Alex Stokes b59c5d6ca1
Add "msgio" functions 2019-08-24 22:37:44 +02:00
Alex Stokes 26165b0729
[wip] sketch of secio handshake 2019-08-24 22:37:44 +02:00
Alex Stokes fd08bcf624
Add `secio` protobufs 2019-08-24 22:37:43 +02:00
Alex Stokes 23f53ef954
Allow optional peer ID in a security session 2019-08-24 22:37:43 +02:00
Alex Stokes 27c0a4f77b
formatting 2019-08-24 22:37:43 +02:00
Alex Stokes 3e07faf343
Merge pull request #266 from ralexstokes/remove-unrelated-cde
Remove test suite for `asyncio`, *not* `py-libp2p`
2019-08-24 22:22:06 +02:00
Alex Stokes d764ca2884
Merge pull request #265 from ralexstokes/add-wait-closed
Add call to `wait_closed` method of asyncio.StreamWriter
2019-08-24 22:17:29 +02:00
Alex Stokes 7ccdeaf308
Remove test suite for `asyncio`, *not* `py-libp2p` 2019-08-24 22:10:47 +02:00
Alex Stokes 5b32bfdd3f
Add call to `wait_closed` method of asyncio.StreamWriter 2019-08-24 22:06:24 +02:00
Alex Stokes d9883ee4f0
Merge pull request #263 from ralexstokes/remove-stream-from-connection
Encapsulate the concept of a stream to the stream multiplexer
2019-08-24 21:58:59 +02:00
Alex Stokes 9c5fb4fa5a
Encapsulate concept of a "stream id" to a "muxed" connection 2019-08-24 21:50:07 +02:00
Alex Stokes e29c1507bf
remove unused fields 2019-08-24 21:50:06 +02:00
Alex Stokes 9a74797068
Merge pull request #264 from ralexstokes/remove-simple-security
Removes the SimpleSecurityTransport
2019-08-24 21:49:29 +02:00
Alex Stokes 73495038e1
remove simple security 2019-08-24 21:39:25 +02:00
Alex Stokes 1790e48c99
Remove deprecated file 2019-08-24 21:36:30 +02:00
Kevin Mai-Husan Chia da3c8be464
Merge pull request #253 from mhchia/feature/plaintext-2.0.0
Add `/plaintext/2.0.0` secure channel
2019-08-22 23:40:02 +08:00
mhchia b516579256
Remove the unnecessary RSAPrivateKey.from_bytes 2019-08-22 22:54:14 +08:00
mhchia c1eacf221f
PR feedback
- Check if the received peer id matches the one we initialize the
session with.
- Move the check inside `run_handshake`
2019-08-22 22:53:49 +08:00
Kevin Mai-Husan Chia 7c630df610
Update libp2p/security/insecure/transport.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-22 22:53:49 +08:00
mhchia 16a4fd33c1
PR feedbacks
- Move exceptions to exceptions.py
- Raise `UpgradeFailure` in upgrader
- Refine the try/catch for upgraders in swarm
2019-08-22 22:53:49 +08:00
mhchia 3e04480d62
Raise `HandshakeFailure` in transport
Change the exception handling flow.
Raise `SecurityUpgradeFailure` in security_multistream.
2019-08-22 22:53:48 +08:00
mhchia 80452d9589
Fix `make_exchange_message` to use the new API 2019-08-22 22:53:48 +08:00
mhchia 921bfb65cc
Verify the remote pubkey and peer_id
- Add `from_bytes` in RSAPublicKey and Secp256k1PublicKey
- Add `pubkey_from_protobuf` to parse pubkey from protobuf
- Verify key and peer_id in `InsecureSession.run_handshake`
2019-08-22 22:53:48 +08:00
mhchia ef476e555b
Use RawConnection.read
Instead of accessing its reader and writer directly.

TODO: considering add `ReaderWriterCloser` interface and let connection
and stream inherit from it.
2019-08-22 22:53:48 +08:00
mhchia 0b466ddc86
Add lock to `RawConnection`
To avoid `self.writer.drain()` is called in parallel.
Reference: https://bugs.python.org/issue29930
2019-08-22 22:53:47 +08:00
mhchia 5768daa9bf
PR feedbacks
- Nits
- Add `SecurityUpgradeFailure` and handle `UpgradeFailure` in Swarm.
2019-08-22 22:53:47 +08:00
Kevin Mai-Husan Chia 2a1367b011
Apply suggestions from code review
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-22 22:53:47 +08:00
mhchia 0b62321265
Fix `test_security_multistream`
By passing initiator keypairs to node.
2019-08-22 22:53:47 +08:00
mhchia de8d356955
Fix tests failure due to lack of peer id
Fix it through doing plaintext handshake.
2019-08-22 22:53:47 +08:00
mhchia bb7d37fd4f
Fix msg encoding
- Change varint-prefix encode to fixedint-prefix(4 bytes) encode.
2019-08-22 22:53:46 +08:00
mhchia 22b1a5395d
A working plaintext 2.0 without validation 2019-08-22 22:53:46 +08:00
mhchia a0923d202a
Move varint and delim read/write to toplevel
To `libp2p.utils`.
2019-08-22 22:53:46 +08:00
mhchia 5192944724
Update pb 2019-08-22 22:53:46 +08:00
mhchia 59b373b48a
Add `plaintext.proto`
Update Makefile to handle the import in `plaintext.proto`.
Import path is modified to be relative to the project root.
And we run `protoc` from where `Makefile` locates, i.e. the project
root.

Reference:
- plaintext.proto: 62b2c6c482/sec/insecure/pb/plaintext.proto
2019-08-22 22:53:46 +08:00
mhchia 7bc363f2fa
Remove initiator in `Mplex`
Besides, fix the wrong passed `multi_addr` to `mplex_stream`.
2019-08-22 22:53:45 +08:00
Kevin Mai-Husan Chia 8217319c28
Merge pull request #258 from NIC619/adjust_coincurve_version_to_match_trinity
Adjust coincurve version to match Trinity's
2019-08-22 22:19:25 +08:00
NIC619 87f5a86950
Adjust coincurve version 2019-08-22 18:01:41 +08:00
Kevin Mai-Husan Chia 46b4a446c3
Merge pull request #257 from mhchia/fix/add-missing-initpy
Fix the issue when importing from other package
2019-08-22 16:32:34 +08:00
mhchia 2752e7a04b
Ignore mypy error here 2019-08-22 15:58:38 +08:00
mhchia dbd4ed44dd
Fix the issue when importing from other package 2019-08-22 15:36:17 +08:00
Kevin Mai-Husan Chia 6b05d9ab8f
Merge pull request #252 from mhchia/fix/add-negotiation-when-upgrading-to-mplex
Negotiate multiplexer protocol when upgrading to `MuxedConn`
2019-08-21 12:30:22 +08:00
mhchia 550c23f9f9
PR feedback
- Use the order in `MuxerMultistream` as the precedence in multiselect
2019-08-21 11:43:25 +08:00
mhchia 8596f7390f
PR feedback: set protocol_id to constants 2019-08-21 11:43:24 +08:00
mhchia d7d8440b2c
PR feedback: nitpicks 2019-08-21 11:43:24 +08:00
mhchia 4358a4bc89
Negotiate multiselect version for Muxer
`MuxerMultistream` is introduced to negotiate `Multiselect` version
before negotiating Multiplexer's version. This is required by
multistream 1.x
2019-08-21 11:43:24 +08:00
Alex Stokes 20aed4430e
Merge pull request #256 from ralexstokes/patch-up-key-to-bytes
Internalize the protobuf serialization to the concept of a `Key`
2019-08-20 10:33:33 -07:00
Alex Stokes e1d3f1601f
Satisfy mypy 2019-08-20 19:28:32 +02:00
Alex Stokes 87d943aa39
Internalize the protobuf serialization to the concept of a `Key`
Given its use across various components of `libp2p` (not just peer IDs),
it makes the abstraction cleaner to pull the serialization into the
key class and expose the canonical serialization to bytes.
2019-08-20 19:01:36 +02:00
mhchia 5d611801c7 Fix isort 2019-08-18 19:51:04 +08:00
mhchia 86d4ce1da8 Add `delim_encode` and `delim_read`
- Add `StreamCommunicator` and `RawConnectionCommunicator`, read/write
messages with delim codec, with `IMuxedStream` and `IRawConnection`
respectively.
- Use it in `Multiselect` and `MultiselectClient`.
2019-08-18 19:51:04 +08:00
mhchia 8cd23abfe2 Remove the leftover merge related stuff 2019-08-16 11:03:16 +08:00
mhchia e293b89925 Fix mypy
It seems the stub doesn't allow default value for params
2019-08-16 11:03:16 +08:00
mhchia c5f32bf431 PR feedback for `MplexStream.read` 2019-08-16 11:03:16 +08:00
Kevin Mai-Husan Chia 8699568d43 Update libp2p/stream_muxer/mplex/mplex_stream.py
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
2019-08-16 11:03:16 +08:00
mhchia f2c31f6fe3 Fix isort 2019-08-16 11:03:16 +08:00
mhchia b27cd0f24f Use `bytearray` over `bytes`
To avoid copies.
2019-08-16 11:03:16 +08:00
mhchia 92320523d5 Add the missing exceptions.py 2019-08-16 11:03:16 +08:00
mhchia e37b8bcf19 mypy: Add `read_buffer_nonblocking` in Mplex 2019-08-16 11:03:16 +08:00
mhchia f281e3e1db flake8 2019-08-16 11:03:16 +08:00
mhchia 9f8276fa84 Support `read(n=-1)`
Now, `n=-1` indicates that we want to read until EOF. However, now we
only read until we have no new message.
2019-08-16 11:03:16 +08:00
mhchia 9cb6ec1c48 Modify the behavior of `MplexStream.read` 2019-08-16 11:03:16 +08:00
mhchia 2485a00e24 Modify `NetStream` to read n bytes 2019-08-16 11:03:16 +08:00
Alex Stokes dbdbcf7440
Merge pull request #250 from ralexstokes/add-secio-groundwork
Add secio groundwork
2019-08-15 19:32:17 -07:00
Alex Stokes 7535a02da7
Clean up key gen 2019-08-15 19:24:30 -07:00
Alex Stokes d17e6f3392
Fix some test imports that got botched in rebase 2019-08-15 16:46:23 -07:00
Alex Stokes 9a4e23a803
mypy protobuf plugin requires keyword-based initializers 2019-08-15 16:36:32 -07:00
Alex Stokes 82bae341a7
Run isort over files that were missing it 2019-08-15 16:33:35 -07:00
Alex Stokes 2e3ffb9d53
Use types for {Private,Public}Key and address other missing type hints 2019-08-15 16:33:34 -07:00
Alex Stokes e7d2681fc0
Move base implementations into `BaseSession` 2019-08-15 16:33:34 -07:00
Alex Stokes cda74dd382
Update tests for new logic 2019-08-15 16:33:34 -07:00
Alex Stokes 9e18d7561d
Supply local priv and pub key when upgrading to a secure transport 2019-08-15 16:33:34 -07:00
Alex Stokes 7942b7eaa7
Expose writer 2019-08-15 16:33:34 -07:00
Alex Stokes 20dd7d777a
More efficiently remove trailing newline from message 2019-08-15 16:33:34 -07:00
Alex Stokes 0ebc8ffb21
Wire some missing properties up 2019-08-15 16:33:34 -07:00
Alex Stokes 02e073d85a
Keep the host's private key for use in transports 2019-08-15 16:33:33 -07:00
Alex Stokes 879cbf1abd
Add an "insecure session" that satisfies the `ISecureConn` interface 2019-08-15 16:33:33 -07:00
Alex Stokes fb43728661
Mark some slow tests as such 2019-08-15 16:33:33 -07:00
Alex Stokes 1e5357a1e1
Update the `ISecureConn` interface following the reference and simplify accordingly 2019-08-15 16:33:33 -07:00
Alex Stokes ab7653526f
Code cleanup / formatting 2019-08-15 16:33:33 -07:00
Alex Stokes b98025c379
Move security transports into their respective sub-packages 2019-08-15 16:33:33 -07:00
Alex Stokes d50e1b6872
Use direct types over indirect types 2019-08-15 16:33:32 -07:00
Alex Stokes ff5eaf2429
Merge pull request #249 from ralexstokes/fix-mypy-protos
Add `mypy` protobufs plugin and regenerate protobufs
2019-08-15 16:15:26 -07:00
Alex Stokes b0e57c2be4
Merge pull request #245 from mhchia/fix/add-initpy-pb
Add the missing __init__.py in pb/
2019-08-15 16:04:09 -07:00
Alex Stokes 0f81ca42a6
Add `mypy` protobufs plugin and regenerate protobufs 2019-08-15 16:01:44 -07:00
mhchia 97308c897e
Add the missing __init__.py in pb/
Error: https://circleci.com/gh/mhchia/trinity/5342?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
2019-08-15 21:03:45 +08:00
Alex Stokes 125c5d8e2c
Adjust inheritance so that the MRO is clear for the chain of keys 2019-08-14 09:30:23 -07:00
Alex Stokes 3b19104284
Add missing `ABC` declaration following PR feedback in #240. 2019-08-14 09:25:54 -07:00
Alex Stokes 9977933fd1
Merge pull request #240 from ralexstokes/allow-multiple-identity-types
Allow multiple peer identity types (via different cryptosystems)
2019-08-14 09:13:02 -07:00
Alex Stokes 08e6f2a30c
Update libp2p/crypto/secp256k1.py
pass the secret on to `coincurve` lib

Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-14 09:02:06 -07:00
Alex Stokes 53e583a068
Update libp2p/crypto/secp256k1.py
Pass the secret on to `coincurve` lib

Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-14 09:02:06 -07:00
Alex Stokes 67744bcb0f
Add a basic interop test for Go and Python peer IDs 2019-08-14 09:00:40 -07:00
Alex Stokes ad20d8cb00
Make a `KeyPair` dataclass for passing around key pairs 2019-08-14 09:00:40 -07:00
Alex Stokes 329bd4eb0f
Clean up peer ID tests 2019-08-14 09:00:40 -07:00
Alex Stokes 6506079a36
Generate peer IDs according to the spec 2019-08-14 09:00:39 -07:00
Alex Stokes f9e859a8e9
Clean up peer.ID 2019-08-14 09:00:39 -07:00
Alex Stokes c40314a043
Use new RSA key type 2019-08-14 09:00:39 -07:00
Alex Stokes 61f78c8feb
Add abstraction for a cryptographic key 2019-08-14 09:00:39 -07:00
Alex Stokes 2c68814bae
Add `coincurve` dep for `secp256k1` keys 2019-08-14 09:00:39 -07:00
Alex Stokes 3736592a7a
Merge pull request #241 from ralexstokes/remove-codecov-ci
Remove code coverage check from CI
2019-08-14 08:58:24 -07:00
Alex Stokes dc262ddb58
Remove badge from README 2019-08-14 08:49:38 -07:00
Alex Stokes 9bff7b8ebf
Remove code coverage check from CI 2019-08-13 21:25:22 -07:00
Alex Stokes 992585852f
Use explicit imports in top-level declaration 2019-08-13 17:43:21 -07:00
Alex Stokes 3debd2c808
Run `black` and `isort` w/ the new config 2019-08-13 14:36:42 -07:00
Alex Stokes 87375e0f23
Use the default line length for `black` ('no configuration')
and update `isort` to a stable configuration given the black line length
2019-08-13 14:36:05 -07:00
Alex Stokes a937c6f3fa
Enable `mypy` in Makefile 2019-08-13 14:32:54 -07:00
Chih Cheng Liang 28f6de37ee
Fix the rest of the typing hints (#232)
* ignore kad

* fix swarm, and minor

* fix init and swarm

* ignore pb

* enable mypy

* fix basic host

* fix tcp

* fix mplex

* add typing for pb

* skip format pyi

* [mypy] no need to ignore pb now

* add typing to chat
2019-08-11 16:47:54 +08:00
Chih Cheng Liang dbb702548f add NegotiableTransport type 2019-08-08 16:09:02 +08:00
Chih Cheng Liang 9851ee01fb sort import 2019-08-08 16:09:02 +08:00
Chih Cheng Liang c536aa3e07 flake8 2019-08-08 16:09:02 +08:00
Chih Cheng Liang 5903012e0e add typing to protocol_muxer 2019-08-08 16:09:02 +08:00
NIC Lin 0d709364f8
Merge pull request #226 from NIC619/add_msg_validator
Add topic message validator
2019-08-07 12:10:45 +08:00
NIC619 a1dc68ab70
Apply PR feedback:
add validation failed test to `push_msg` test
2019-08-07 11:53:54 +08:00
NIC Lin b26426214e
Update libp2p/pubsub/pubsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-08-07 11:43:32 +08:00
NIC619 d4febea469
Message was not enforced to carry signature yet 2019-08-06 13:05:31 +08:00
NIC619 1cea1264a4
Raise exception when topic validation failed 2019-08-06 12:38:31 +08:00
NIC619 9a1e5fe813
Add `ValidationError` 2019-08-06 12:37:34 +08:00
NIC619 47643a67c6
Apply PR feedback 2019-08-06 12:32:18 +08:00
NIC619 b96ef0e6c7
Fix:
`_is_subscribed_to_msg` need only subscribe to one of the topics
2019-08-05 18:20:04 +08:00
NIC619 a2efd03dfa
Schedule `push_msg` into a task 2019-08-05 18:19:32 +08:00
NIC619 2bb7f42c20
Add validators to `push_msg` 2019-08-05 18:19:32 +08:00
NIC619 19ce5bb420
Add `signature_validator` stub and docstring 2019-08-05 18:19:32 +08:00
NIC619 e1b86904e3
Add `validate_msg` and test 2019-08-05 18:19:32 +08:00
NIC619 ec2c566e5a
Fix validator return type and add docstring 2019-08-05 18:19:32 +08:00
NIC619 f8ca4fa1ef
Add `get_msg_validators` and test 2019-08-05 18:19:32 +08:00
NIC619 1ed14d0cc8
Add `remove_topic_validator` test 2019-08-05 18:19:32 +08:00
NIC619 cf69f7e800
Rename to `set_topic_validator` and add test 2019-08-05 18:19:31 +08:00
NIC619 b1f4813195
Add add/remove topic validator functions 2019-08-05 18:19:31 +08:00
NIC619 3973f1d13c
Add `pubsub.topic_validators` 2019-08-05 18:19:31 +08:00
NIC Lin ccf4b62976
Merge pull request #203 from ChihChengLiang/tranport-typing
add typing to transport and stream_muxer
2019-08-05 17:19:11 +08:00
Chih Cheng Liang cb3a59e0ac
ttl as int 2019-08-05 17:02:18 +08:00
Chih Cheng Liang 63c733c3f5
PR feedback 2019-08-05 16:58:34 +08:00
Chih Cheng Liang ccfb6eb35f
remove constructor of TCP 2019-08-05 16:56:56 +08:00
Chih Cheng Liang e763f57930
run isort 2019-08-05 11:47:23 +08:00
Chih Cheng Liang 7a04ebb51f
run black 2019-08-05 11:21:20 +08:00
Chih Cheng Liang 9e0a806218
move stream and connection interfaces to abc 2019-08-05 11:17:38 +08:00
Chih Cheng Liang c804f5ad19
minor 2019-08-05 10:47:59 +08:00
Chih Cheng Liang 29091266fc
add still needed TYPE_CHECK 2019-08-05 10:46:49 +08:00
Chih Cheng Liang 87ef2e4618
remove if TYPE_CHECKING as much as possible 2019-08-05 10:46:49 +08:00
Chih Cheng Liang 4c9a930f84
stream_muxer done 2019-08-05 10:45:47 +08:00
Chih Cheng Liang dadac423f2
typed muxed_connection_interface.py 2019-08-05 10:45:47 +08:00
Chih Cheng Liang b64ed9fd6f
typed mplex.utils 2019-08-05 10:45:46 +08:00
Chih Cheng Liang 239a5c88fb
add typing to mplex 2019-08-05 10:45:46 +08:00
Chih Cheng Liang 36b7e8ded9
Refactor HeaderTags 2019-08-05 10:45:46 +08:00
Chih Cheng Liang 29fbb9e40a
add typing to transport 2019-08-05 10:42:43 +08:00
Alex Stokes a20c172480 update isort line length 2019-08-04 12:37:41 +08:00
Alex Stokes 7477b29508 run black w/ extended line length 2019-08-04 12:37:41 +08:00
Alex Stokes 905dfa9a8d Remove dependence on `make` in tox CI run 2019-08-04 12:37:41 +08:00
Alex Stokes 251422a234 Match `black` line length to `flake8` line length 2019-08-04 12:37:41 +08:00
Alex Stokes cb301fcc51 Opt-out of linting on the special cases we have, given new ignore rules 2019-08-04 12:37:27 +08:00
Alex Stokes 03b1304fe7 Remove rules we want to enforce or are unnecessary with our style 2019-08-04 12:37:27 +08:00
Alex Stokes a8acbb72c2
Merge pull request #223 from ralexstokes/remove-grpc
Remove `grpc` dependency
2019-08-03 12:28:38 -07:00
Alex Stokes 986da458e9
Directly use `protobuf` dep 2019-08-03 12:17:09 -07:00
Alex Stokes 69ec86c871
Remove `grpc` dependency 2019-08-03 11:17:28 -07:00
Alex Stokes 910c3fa6f1
Merge pull request #221 from mhchia/feature/add-flake8-and-change-cfg
Add flake8 and change flake8 configs
2019-08-03 11:13:58 -07:00
Alex Stokes cf3904a56a
Merge pull request #207 from ralexstokes/update-chat-example
Update chat example
2019-08-03 09:49:51 -07:00
mhchia 0b11e32000
Remove the `TODO` flag for flake8 2019-08-04 00:32:32 +08:00
mhchia 1cc7e38846
Add flake8 to lint deps 2019-08-04 00:27:02 +08:00
mhchia 0a5b4a88ca
Fix flake8 for the existing code 2019-08-04 00:18:30 +08:00
mhchia 727342a767
Move flake8 settings to tox.ini
And add flake8 in the Makefile.
2019-08-04 00:11:49 +08:00
Arun Babu Neelicattu f20b78c93f
Replace pylint with flake8
This change removes the use of pylint and implements similar
code style checks. This is complementary with black code formatter.
2019-08-04 00:06:34 +08:00
mhchia 4fef80595c Skip pb files in linters
In isort, black, and flake8, respectively.

Add `format` in Makefile

Run top level packages only

With `setup.py`, to avoid running other unrelated directories.

Refactor
2019-08-03 23:34:56 +08:00
Alex Stokes c8005c8113 Run `isort` in repo 2019-08-03 17:50:14 +08:00
Alex Stokes a92d933ed2 Add `isort` to CI check 2019-08-03 17:50:14 +08:00
Alex Stokes d78e6dbf04 Add `black`-compatible `isort` config 2019-08-03 17:50:14 +08:00
Alex Stokes 201850397a Add `isort` as a lint dep 2019-08-03 17:50:14 +08:00
Alex Stokes 21e013e753
Merge pull request #214 from ralexstokes/remove-third-party-key-in-id
Refactor ID to not use third-party type for cryptographic keys
2019-08-02 22:50:53 -07:00
Alex Stokes 6090d2ca3b
Clean up old comments, print stmts, formatting, etc. 2019-08-02 21:58:31 -07:00
Alex Stokes bd8d45fbc1
Refactor ID to not use third-party type for cryptographic keys
Remove `ID.from_privkey` which would require specific knowledge per cryptosystem
2019-08-02 21:51:16 -07:00
Alex Stokes cff5fe0d5f
Add `pytest` config 2019-08-02 15:58:39 -07:00
Alex Stokes 28e1a03dc4
remove empty yamux impl 2019-08-02 15:03:59 -07:00
Alex Stokes d47cddee24
Clean up extraneous files in `tests` 2019-08-02 14:54:40 -07:00
Alex Stokes 224b54ad93
Add basic flake8 config 2019-08-02 14:38:03 -07:00
Alex Stokes 2dfd7794b6
add note about protoc version and add Make command to generate protobuf files 2019-08-02 12:20:48 -07:00
Alex Stokes e55d9f2e60
restore generated protobuf files 2019-08-02 12:09:27 -07:00
Alex Stokes 88a3a3159e
Add clarifying comment about `InsecureTransport` 2019-08-02 12:07:35 -07:00
Alex Stokes b2367e35d8
Merge pull request #208 from ralexstokes/disable-mypy-ci
disable mypy check during CI temporarily and move `black --check` to Makefile
2019-08-02 12:01:47 -07:00
Alex Stokes 7b7c8ad30d
run black over dangling files 2019-08-02 11:47:08 -07:00
Alex Stokes da9d5cadec
Disables `mypy` in CI
Also moves linting to Makefile to take advantage of globbing.
2019-08-02 11:46:43 -07:00
Alex Stokes 612330d318
Merge pull request #205 from mhchia/fix/remove-pylint-leftover
Remove pylint related lines and files
2019-08-02 10:50:44 -07:00
Alex Stokes c8d175b373
Add a localhost option and fix the printed example to run another peer 2019-08-02 10:22:15 -07:00
Alex Stokes 430b4e2f89
Bail as soon as we know there is a port error 2019-08-02 10:21:41 -07:00
mhchia 2e94fcf56c
Remove pylint:disable 2019-08-02 23:26:06 +08:00
mhchia 06a9511ab4
Remove .pylintrc 2019-08-02 23:20:30 +08:00
NIC Lin 7a0fa7dd37
Merge pull request #196 from NIC619/fix_peer_id
Refactor `peer.ID` class and only takes in `bytes` type argument
2019-08-02 18:03:56 +08:00
NIC619 568454534f
Remove unused ID type conversions 2019-08-02 16:48:38 +08:00
NIC619 ee290b2ac2
Fix missing asterisk 2019-08-02 16:48:32 +08:00
NIC619 3a4b592a5a
Fix missing asterisk 2019-08-02 15:06:39 +08:00
Kevin Mai-Husan Chia 38423cc2a4
Merge pull request #200 from ChihChengLiang/add-typing-security
Add typing to security module
2019-08-02 14:57:24 +08:00
NIC619 0173b5e0d9
Add ID.to_base58() and comparison against str type 2019-08-02 14:46:01 +08:00
NIC619 924e965537
Change argument name of `Network.listen` and blakc format 2019-08-02 14:45:59 +08:00
NIC619 cd684aad9e
Update peer_id to type peer.ID in pubsub folder 2019-08-02 14:45:23 +08:00
NIC619 9562cb2a46
Rename:
`KadPeerInfo.peer_id` to `KadPeerInfo.peer_id_bytes`
2019-08-02 14:43:01 +08:00
NIC619 f00e80bc25
Fix wrong peer id type used in KadPeerInfo 2019-08-02 14:42:10 +08:00
NIC619 b928bdb356
Convert from base58/pubkey/privkey to class method 2019-08-02 14:42:10 +08:00
NIC619 80481252ca
Refactor ID to take in type bytes only 2019-08-02 14:42:10 +08:00
Chih Cheng Liang 10a8347c6a
PR feedback 2019-08-02 14:12:59 +08:00
Kevin Mai-Husan Chia f6e456c96e
Merge pull request #198 from mhchia/fix/clean-up-tests-with-fixture
Use factories and fixtures in pubsub tests
2019-08-01 22:08:03 +08:00
mhchia 716c60ca6d
Reflect PR feedback
- Remove leftover `int` in `GossipsubParams`
- Remove default fields in tests
2019-08-01 21:38:14 +08:00
Chih Cheng Liang e731f77f2d
minor 2019-08-01 19:13:43 +08:00
Chih Cheng Liang a86f010c95
add typing to security 2019-08-01 19:12:11 +08:00
Chih Cheng Liang 030abcc959
add vscode 2019-08-01 17:55:15 +08:00
mhchia 33d233f5df
Add missing comma 2019-08-01 13:31:02 +08:00
mhchia 4eb846be7c
Remove the assertions for `connect`
Because it fails in `test_security_multistream.py` and this check is
indeed not necessary inside `connect`.(Should probably be outside the
function.)
2019-08-01 13:30:09 +08:00
mhchia c72dfe1dd3
Use factories and fixtures in pubsub tests
Done
- Add factories using factory-boy
- Modify fixtures and tests to use factories
- Modify tests to use fixtures and factories
- Clean up
2019-08-01 13:30:08 +08:00
Kevin Mai-Husan Chia 9181cf95f0
Merge pull request #199 from ralexstokes/add-black-to-repo
Add `black` to repo
2019-08-01 10:53:44 +08:00
Alex Stokes 51cc710dc0
remove pylint 2019-07-31 15:14:56 -07:00
Alex Stokes 0ae9840928
Run `black` over repo 2019-07-31 15:00:12 -07:00
Alex Stokes a2133d8c7c
Add `black` check to CI 2019-07-31 14:59:47 -07:00
Alex Stokes 936369aa5e
Add `black` as a lint dependency 2019-07-31 14:53:28 -07:00
Kevin Mai-Husan Chia 1727ba48d9
Merge pull request #190 from mhchia/feature/pubsub-test
Add `test_pubsub.py`
2019-07-31 16:40:38 +08:00
mhchia ce369d47e9
Refine exception message 2019-07-31 16:23:07 +08:00
Kevin Mai-Husan Chia c0e253a524
Update tests/pubsub/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-31 16:07:48 +08:00
Kevin Mai-Husan Chia 3bb63612a9
Update tests/pubsub/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-31 16:07:04 +08:00
mhchia 9683d5e8ac
Add tests for `Pubsub`
- `test_handle_subscription`
- `test_handle_talk`
- `test_message_all_peers`
2019-07-31 14:15:11 +08:00
mhchia 3a42d72cd9
Fix a minor bug for pb optional field
In `Pubsub.continuously_read_stream`, it checks whether this is a
control message enclosed in RPC message with `if rpc_incoming.control:`.
However, in pb2, the condition is always true because a default value is
returned when a field is not set. Solved it by changing it to
`if rpc_incoming.HasField("control"):`.
2019-07-31 14:15:11 +08:00
mhchia 037b95252d
Add tests for `Pubsub`
- `test_get_hello_packet`
- `test_continuously_read_stream`
- `test_publish`
- `test_push_msg`
2019-07-31 14:15:10 +08:00
mhchia 550289a439
Combine test_subscription.py to test_pubsub.py
And add a bunch of tests for pubsub
2019-07-31 14:15:10 +08:00
mhchia 96563c0d84
Add fixtures for pubsub and router
And a starting `test_pubsub.py`
2019-07-31 14:14:13 +08:00
NIC Lin 21e97407ef
Merge pull request #192 from NIC619/add_type_hint
Add type hints to host/, peer/, network/, pubsub/, routing/
2019-07-31 11:13:37 +08:00
NIC619 5e215901c0
Apply PR feedback 2019-07-30 23:41:28 +08:00
NIC619 76de01a17d
Add `duplicate-code` and `cyclic-import` to pylintrc 2019-07-30 18:01:01 +08:00
NIC619 437b7665c4
Fix:
type object not subscriptable
2019-07-30 18:00:30 +08:00
NIC619 60d6703964
Temporary disable pylint on tests folder 2019-07-30 17:41:46 +08:00
NIC619 c4105688d1
Fix after rebase 2019-07-30 17:31:08 +08:00
NIC619 e53727d301
Apply PR feedback: fix type hints 2019-07-30 16:28:25 +08:00
NIC619 2d4e23cfe2
Fix cyclic import and lint 2019-07-30 16:28:05 +08:00
NIC619 a4a0d79f6d
Improve import layout 2019-07-30 16:27:29 +08:00
NIC619 e7ac09cb94
Fix:
Add Gossipsub attribute `peers_protocol` and do cleanup when peer removed
2019-07-30 16:26:21 +08:00
NIC619 d716e90e17
Fix on type hints 2019-07-30 16:25:33 +08:00
NIC619 edd164c878
Add type hints to network folder 2019-07-30 16:25:17 +08:00
NIC619 e1592997a8
Add type hints to routing folder 2019-07-30 16:24:52 +08:00
NIC619 f2de986c74
Add type hints to peer folder 2019-07-30 16:24:51 +08:00
NIC619 b695b0e1ec
Add type hint to host folder 2019-07-30 16:24:34 +08:00
NIC619 b2f496d081
Fix type hints except pb msg in pubsub folder 2019-07-30 16:24:34 +08:00
NIC619 a0aa105867
Add type hint to pubsub notifee/interface 2019-07-30 16:23:15 +08:00
NIC619 3549f2ff8b
Add type hints to mcache.py 2019-07-30 16:20:48 +08:00
NIC619 63014eeaae
Add type hints to floodsub.py 2019-07-30 16:20:47 +08:00
NIC619 b920955db6
Add type hints to gossipsub.py 2019-07-30 15:39:50 +08:00
NIC619 8eb6a230ff
Fix and add type hints to pubsub.py 2019-07-30 15:32:58 +08:00
Kevin Mai-Husan Chia f0046fa3e0
Merge pull request #187 from mhchia/feature/pubsub-publish
Add `Pubsub.publish`
2019-07-29 12:29:34 +08:00
mhchia f02d38c0ee
Reflect PR feedback
* Rename `src` to `msg_forwarder` in pubsub/floodsub/gossipsub
* Rename Variables
* Sort imports
* Clean up
2019-07-29 12:09:35 +08:00
mhchia 74d831d4e2
Reflect PR feedback 2019-07-28 18:06:38 +08:00
Kevin Mai-Husan Chia 70c5c84f32
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:09:01 +08:00
Kevin Mai-Husan Chia a1e20caebe
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:07:11 +08:00
Kevin Mai-Husan Chia ffb3920468
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:06:03 +08:00
Kevin Mai-Husan Chia c252c62009
Update libp2p/pubsub/pubsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:05:29 +08:00
mhchia 766d8ba1e1
A little bit clean up 2019-07-27 12:06:36 +08:00
mhchia c028aef2de
Fix all tests
- Dedup `perform_test_from_obj` and the test cases used in both
`test_floodsub` and `test_gossipsub_backward_compatibility.py`.
Therefore, they are put in the standalone file
`tests/pubsub/floodsub_integration_test_settings.py`. The functions
and testcases are imported from there then.
- IMO still need a refactor on the tests. There are still some duplicate
code.
2019-07-27 11:49:03 +08:00
mhchia 65aedcb25a
Fix several tests 2019-07-26 18:35:25 +08:00
mhchia 035d08b8bd
Fix `test_floodsub.py` 2019-07-25 23:11:27 +08:00
mhchia dadcf8138e
Fix the tests according to `pubsub.Publish`
And refactored a bit.
2019-07-25 16:58:00 +08:00
mhchia cae4f34034
Refactor `floodsub.publish`
Passed the first test of floodsub
2019-07-25 14:08:16 +08:00
mhchia 93cf5a2c32
A roughly skeleton of `floodsub.publish`
Still need to ensure when to deliver to ourselves
2019-07-24 22:33:32 +08:00
mhchia 3f52b0dc0a
Remove leftover imports 2019-07-24 21:57:46 +08:00
mhchia b528c211b9
Temp modified publish 2019-07-24 21:55:57 +08:00
mhchia 218bdb42c4
Add basic functionalities of `publish` 2019-07-24 21:55:04 +08:00
mhchia 8f9e5a28ff
Temp 2019-07-24 21:54:36 +08:00
Kevin Mai-Husan Chia f329c5a627
Merge pull request #188 from mhchia/feature/add-typing-for-pubusb
Add tox and mypy
2019-07-24 21:45:19 +08:00
mhchia a97bac0a02
Remove sudo from .travis.yml 2019-07-24 21:30:23 +08:00
mhchia d64c7d6d56
Add the missing type for `port` 2019-07-24 21:28:14 +08:00
mhchia ecf4e373da
Remove unused `aio_timers` 2019-07-24 18:49:51 +08:00
mhchia 381f5ddc3a
Replace `click` with `argparse` 2019-07-24 18:43:49 +08:00
mhchia 04b7df9fcf
Lint `examples` in tox 2019-07-24 18:00:57 +08:00
mhchia 529829b9f1
Move `codecov` to tox.ini 2019-07-24 16:41:19 +08:00
mhchia 1ae306ae8f
Fix mypy command
- Remove requirements_dev.txt
- Add detailed versions
2019-07-24 16:34:55 +08:00
mhchia d3a948be47
Fix error: Change params floodsub.publish back 2019-07-24 16:24:14 +08:00
mhchia 8da4032c3b
Let pylint not complain about FIXME, XXX 2019-07-24 16:23:30 +08:00
mhchia e428897cc8
Add the missing tox in CI 2019-07-24 16:11:05 +08:00
mhchia 859ec6e241
Add the missing env for py37-test 2019-07-24 16:09:03 +08:00
mhchia ae4c135ae1
Change travis CI config 2019-07-24 16:05:33 +08:00
mhchia 9497c3180f
Add tox
- Put extras_require to setup.py
- Add mypy
2019-07-24 15:54:30 +08:00
mhchia d6c19e71a6
Add typing and notes in pubsub 2019-07-24 14:54:30 +08:00
NIC Lin 4819959e5a
Merge pull request #184 from NIC619/fix_handle_graft
Fix `gossipsub.handle_graft`
2019-07-24 13:55:07 +08:00
NIC619 b0b4ddd0ca
Fix `test_handle_prune` 2019-07-24 11:35:14 +08:00
NIC619 99eabe49eb
Add `handle_prune` test 2019-07-23 23:00:43 +08:00
NIC619 99252e49f8
Prevent re-adding peers to mesh 2019-07-23 23:00:43 +08:00
NIC619 42093e40ec
Add `handle_graft` test 2019-07-23 22:53:01 +08:00
NIC619 e52bfe3a51
Fix:
Respond GRAFT with PRUNE if not subscribed to the topic
2019-07-23 22:52:24 +08:00
NIC Lin 84824fd566
Merge pull request #182 from NIC619/fix_refactor_gossipsub_join
Fix and refactor: `gossipsub.join`
2019-07-23 18:12:37 +08:00
NIC619 0cc8a205fb
Fix test and reduce number of nodes in `test_join` 2019-07-23 17:28:46 +08:00
NIC619 c0a3af69e0
Apply PR feedback:
Check that node is in mesh peer's mesh after subscribe
2019-07-23 16:45:54 +08:00
NIC Lin 042e0ac916
Update tests/pubsub/test_gossipsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-07-23 16:37:41 +08:00
NIC Lin 3c3a9ac90b
Update tests/pubsub/test_gossipsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-07-23 16:37:01 +08:00
NIC619 afc853a776
Apply PR feedback 2019-07-22 23:22:07 +08:00
NIC619 cdbeb63879
Add test 2019-07-22 19:28:12 +08:00
NIC619 67f9edb77d
Remove fanout topic after joining the topic 2019-07-22 19:28:07 +08:00
NIC619 4ab99485a6
Fix lint error 2019-07-21 23:32:54 +08:00
NIC619 14ee44c549
Lint test and add cleanup to the end 2019-07-21 23:16:42 +08:00
NIC619 41141c028b
FIx:
check topic exist in `pubsub.peer_topics`
2019-07-21 23:16:41 +08:00
NIC619 2c1c8dc8cf
Add `gossipsub.join` test 2019-07-21 23:16:41 +08:00
NIC619 fd1f318b0c
Fix:
in mesh heartbeat, select from gossipsub peers subscribed to the topic
2019-07-21 23:14:08 +08:00
NIC619 a26c7783d6
Add `one_to_all_connect` 2019-07-21 23:14:08 +08:00
NIC619 404dc67e83
Fix:
prevent selecting peers from topic not in peer topics
2019-07-21 23:14:08 +08:00
NIC619 b5c3420c16
Refactor `gossipsub.join` 2019-07-21 23:14:08 +08:00
NIC Lin 1e78c21eca
Merge pull request #181 from NIC619/fix_handle_unsubscribe
Fix: handle unsubscribe message
2019-07-21 20:09:51 +08:00
Kevin Mai-Husan Chia 69a35536f4
Merge pull request #179 from mhchia/feature/latest-maddr-and-pin-versions
Use the latest multiaddr and pin deps versions
2019-07-20 22:57:35 +08:00
mhchia 8d3b3fdb6c
Add debug msg 2019-07-20 22:47:37 +08:00
Kevin Mai-Husan Chia 085a5e7526
Apply suggestions from code review
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-20 22:43:32 +08:00
mhchia a2d1aadf25
Add test for checking p2p code 2019-07-20 22:43:32 +08:00
mhchia d3eb68fa50
Increase coverage 2019-07-20 22:43:32 +08:00
mhchia 73125b99b0
Fix the error due to the breaking change
In multiaddr, `split` is moved to `Multiaddr`'s method.
2019-07-20 22:43:31 +08:00
mhchia 0fbf45d8ca
Pin the versions
Especially, use the latest multiaddr
2019-07-20 22:42:18 +08:00
Kevin Mai-Husan Chia 4422888f6f
Merge pull request #180 from mhchia/fix/include-all-pkgs-setuptools
Change `packages` in `setup`
2019-07-20 22:39:57 +08:00
NIC Lin d37f7f64e3
Merge pull request #176 from NIC619/small_fix_on_docstring
Fix some docstrings
2019-07-20 22:12:08 +08:00
NIC619 29aae7dca4
Add gossipsub join/leave test 2019-07-19 20:16:53 +08:00
NIC619 36575e8c9b
Add check to prevent gossipsub re-join and re-leave 2019-07-19 19:59:48 +08:00
NIC619 183eee0e85
Add self subscription tests 2019-07-18 13:39:22 +08:00
NIC619 89347be526
Prevent self re-unsubscription 2019-07-18 13:26:31 +08:00
NIC619 f25d97fbd3
Prevent self re-subscription 2019-07-15 16:32:05 +08:00
NIC619 6d9ec7a9c5
Handle the unsubscribe case 2019-07-15 16:28:29 +08:00
mhchia edd02c498f
Ignore pb files 2019-07-11 20:58:03 +08:00
mhchia 31cd520076
Add the missing __init__.py in pb 2019-07-11 17:50:36 +08:00
mhchia a7ce230d05
Change params `multiaddr` to `maddr`
To make pylint happy
2019-07-10 19:33:38 +08:00
mhchia c8cb375d19
Make pylint happy
Not sure why it fails now, but happy previously.
2019-07-10 17:44:14 +08:00
mhchia fa092505e8
Add missing "package file" 2019-07-10 00:07:58 +08:00
mhchia 10511d4930
Clean up a bit 2019-07-09 23:55:43 +08:00
mhchia 0188985918
Change `packages` in `setup`
Previously, only the top level `libp2p` is included.
This left only `__init__.py` in the built distribution.
Use `setuptools.find_packages` with `exclude` instead, to avoid tiringly
list all of the subpackages.
2019-07-09 23:29:03 +08:00
NIC619 900b6d9f37
Fix docstring 2019-06-21 11:51:11 -06:00
Jason Carver a6f6078814
Add vim .swo files and .mypy_cache to .gitignore 2019-06-03 12:22:46 -07:00
stuckinaboot b001256f5f
Merge pull request #166 from libp2p/peer-id
Full-length Peer ID
2019-05-22 12:58:45 -04:00
Stuckinaboot 9340d03660 Merge branch 'peer-id' of https://github.com/libp2p/py-libp2p into peer-id 2019-05-22 11:37:31 -04:00
Stuckinaboot a4529d827d Modify peer_id str 2019-05-22 11:37:07 -04:00
Stuckinaboot 4b40339b63 Update peerid tests for longer peerids 2019-05-22 11:37:07 -04:00
Stuckinaboot ae44c062df Remove truncating peerid when converting to string 2019-05-22 11:37:07 -04:00
stuckinaboot 8bdcc63296
Merge pull request #167 from ralexstokes/pin-multiaddr-version
Pin the version of multiaddr we require.
2019-05-22 11:33:49 -04:00
Stuckinaboot f9721a2ade Modify peer_id str 2019-05-21 22:25:08 -04:00
Alex Stokes c934100526
Pin the version of multiaddr we require.
Temporary workaround for https://github.com/multiformats/py-multiaddr/issues/47
2019-05-16 11:15:23 -04:00
Stuckinaboot 1250182e7c Update peerid tests for longer peerids 2019-05-08 20:19:13 -04:00
Stuckinaboot 702af198f2 Remove truncating peerid when converting to string 2019-05-08 20:19:02 -04:00
ZX daec48b601
Merge pull request #165 from libp2p/green-apples
update green apples
2019-05-07 13:53:54 -04:00
zixuanzh 82682ab03f update green apples 2019-05-07 10:08:41 -04:00
ZX 49384ad3d7
Merge pull request #159 from libp2p/security-dev
Implement security
2019-05-07 09:56:19 -04:00
Stuckinaboot 867f3a70f6 Modify stream to be reader_writer 2019-05-07 01:31:54 -04:00
Robert Zajac 9052e8f8bd
The Gossipsub PR (#162)
* Add handle_rpc call to pubsub

* Scaffold gossipsub functions

* Add timer

* Implement most of mesh construction

* Implement emit and handle

* Implement fanout heartbeat

* Refactor emit

* some gossipsub cleanup and test

* minor lint stuff, more to come

* Implement publish

* Fix comment

* Modify pubsub/gossipsub so that floodsub tests pass using gossipsub router

* Add floodsub tests to gossipsub

* Handle case where select_from_minus, num_to_select > size(pool-minus)

* Add topic membership

* Implement handle ihave

* Implement most of iwant

* Add mcache.add and comments

* Refactor handle_ihave

* Implement stream write in handle_iwant

* Implement gossip heartbeat

* unresolved vars

* initial mcache code

* documenting mcache

* writing test/debugging mcache

* finished mcache test and debugged

* Make gossipsub backward compatibility its own file

* remove mcache prints

* DEBUGGING

* Add sender_peer_id to handle_rpc to get gossip test passing

* Modify gossipsub to make fanout work

* fanout maintenance test

* debugging gsub GOSSIP

* DEBUGGING

* debugged sender seen cachce

* adding lru, removing prints

* pylint cleanup

* Fix github comments in PR

* minor floodsub possible bugfix
2019-05-06 23:44:13 -04:00
Alex Haynes eea6a9fda7
Merge pull request #157 from libp2p/router-refactor
Refactored RoutedHost into Injected Router
2019-05-05 14:45:22 -04:00
Alex Haynes ea4fd8e81f requested changes 2019-05-05 14:32:41 -04:00
stuckinaboot ff500e6d8e
Merge branch 'master' into security-dev 2019-05-01 17:26:23 -04:00
Stuckinaboot 144dc8d854 Move simple security to libp2p/security 2019-05-01 17:21:11 -04:00
Stuckinaboot 515a461172 Add more security tests 2019-05-01 17:13:01 -04:00
Jason Carver 81af1b6444
Merge pull request #26 from pipermerriam/piper/update-to-native-pypy-markdown-support
switch to native pypy markdown support
2019-05-01 13:53:19 -07:00
Piper Merriam 84d2b22a7b
switch to native pypy markdown support 2019-05-01 14:32:48 -06:00
Stuckinaboot 4333c2d061 Fix linting issues' 2019-05-01 13:54:19 -04:00
Alex Haynes 0f5854a83b
Merge pull request #160 from libp2p/providers
implement add get providers
2019-04-30 21:48:17 -04:00
Stuckinaboot e555f17a7b Fix bug in security multistream 2019-04-30 16:07:26 -04:00
Stuckinaboot a0bd6e5eb0 Add simple security with communication test 2019-04-30 03:27:06 -04:00
Stuckinaboot f59f27d4d0 Integrate security selectin into libp2p system 2019-04-30 03:09:05 -04:00
Stuckinaboot 999e64854c Add security tests 2019-04-29 18:05:49 -04:00
Stuckinaboot 1fb4372ede Refine security 2019-04-29 18:05:38 -04:00
zixuanzh e1d6fdae73 pass test 2019-04-28 18:18:56 -04:00
zixuanzh db7be2d561 add simple test 2019-04-28 17:57:57 -04:00
Alex Haynes 2efc0d3b5c updated peerinfo encoding stored in DHT 2019-04-28 17:38:40 -04:00
zixuanzh 8ab387ac05 refactor add_router in swarm 2019-04-28 15:00:43 -04:00
ZX 21a99287b1
Merge branch 'master' into router-refactor 2019-04-28 14:51:54 -04:00
zixuanzh 3b1a3fb198 fix tests 2019-04-28 13:58:57 -04:00
Alex Haynes b9ae73d317
Merge pull request #158 from libp2p/init-refactor
move transport to swarm constructor
2019-04-27 22:59:05 -04:00
Alex Haynes fdaa4f56b9 working on adding tests 2019-04-27 21:59:25 -04:00
Stuckinaboot eb26661652 Remove outdated encryption folder 2019-04-27 20:09:25 -04:00
Stuckinaboot d861a00d60 Scaffold security 2019-04-27 19:42:05 -04:00
Christoph Burgdorf 720b2cf3d2 Add mypy support 2019-04-26 18:22:24 +02:00
zixuanzh 4436886371 implement add get providers 2019-04-25 13:25:09 -04:00
zixuanzh 35f75c4dcd move transport to swarm constructor 2019-04-24 22:36:09 -04:00
Alex Haynes 98f8df25ab Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:26:30 -04:00
Alex Haynes 159301306d Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:26:17 -04:00
Alex Haynes 2b8c7c294a Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:18:03 -04:00
Alex Haynes 7962060cc0 refactored routedhost into router passed to swarm 2019-04-24 22:17:35 -04:00
Alex Haynes 6c5bac53d7 refactored routedhost into router passed to swarm 2019-04-24 22:11:54 -04:00
Alex Haynes fc4fc74b87
Merge pull request #153 from libp2p/routed-host
Added RoutedHost
2019-04-24 21:12:15 -04:00
Alex Haynes c4f8bf6ce1 fixed requested changes 2019-04-24 20:59:22 -04:00
Alex Haynes 73ce3461e2 fixed typo 2019-04-21 13:45:18 -04:00
Alex Haynes 9c5cd05042 updated id access in kadpeerinfo 2019-04-21 13:27:44 -04:00
Alex Haynes 5fc57c1076 removed duplicate code 2019-04-21 13:22:11 -04:00
Robert Zajac db7f890e93
remove unused umsgpack (#155) 2019-04-21 12:16:24 -04:00
Alex Haynes e7424d3673 added RoutedHost and updated new_node to support it 2019-04-20 17:35:05 -04:00
ZX 367a939087
Merge pull request #152 from libp2p/node-refactor
Node refactor
2019-04-19 21:09:44 -04:00
Alex Haynes b0edc47b9a removed commented out code 2019-04-19 21:04:18 -04:00
Alex Haynes dbe3fcbf5a updated same_home_as to work for variable number of addresses 2019-04-19 21:00:55 -04:00
Alex Haynes 3fa6a165b3 Merge branch 'node-refactor' of https://github.com/libp2p/py-libp2p into node-refactor 2019-04-19 20:54:37 -04:00
zixuanzh 69f9aa5f0e refactor KadPeerInfo construction 2019-04-19 20:44:17 -04:00
zixuanzh 1512ae27a1 fix all tests 2019-04-19 20:09:32 -04:00
zixuanzh fb687dad09 refactor peerinfo 2019-04-19 20:09:32 -04:00
zixuanzh c03f2f63d2 replace id with peer_id 2019-04-19 20:09:32 -04:00
zixuanzh 9ddbd18ded replace node with KadPeerInfo
replace node with kadpeerinfo
2019-04-19 20:09:18 -04:00
Robert Zajac 5d4b4cf525
fixing chat (#151) 2019-04-18 15:56:02 -04:00
Alex Haynes cce226c714
Merge pull request #129 from zaibon/peer_routing
[WIP] kademlia dht
2019-04-17 21:50:03 -04:00
Alex Haynes 1228b11bc9
Update test_basic.py 2019-04-17 21:44:45 -04:00
zixuanzh 9906e23b55 fixed linting 2019-04-17 21:39:30 -04:00
zixuanzh 002a46f0d4 add dependencies 2019-04-17 21:25:02 -04:00
ZX 14da7d49fe
Merge branch 'master' into peer_routing 2019-04-17 21:22:37 -04:00
zixuanzh 3c6ad886bb add dependencies 2019-04-17 21:21:17 -04:00
Alex Haynes 3ee75f0ecd updated travis config 2019-04-17 21:03:35 -04:00
Alex Haynes d5c7cc7cb7 added changes from future commits to last passing bmuller commit 2019-04-17 20:21:59 -04:00
Robert Zajac c8908f7e67
keep lru_dict in setup.py only 2019-04-09 20:50:21 -04:00
ZX 629d6dfd3f
Merge pull request #147 from libp2p/lru-cache
LRU cache on seen_messages #146
2019-04-06 12:26:15 -04:00
zixuanzh 7a298adc33 add simple lru test 2019-04-05 21:46:18 -04:00
zixuanzh d04798ce7c lru cache seen_messages 2019-04-05 21:46:03 -04:00
zixuanzh 1f3aaded1d add lru-dict as requirement 2019-04-05 21:45:31 -04:00
stuckinaboot ffd3bac2ef
Merge pull request #144 from libp2p/seqno
Seqno. Addresses seqno and removing unused print statements in #146
2019-04-05 17:50:47 -04:00
Stuckinaboot 686c55b09c Remove unnecessary print #146 2019-04-05 17:38:29 -04:00
Stuckinaboot b4aa64bc39 Change >I to >Q for 64-bit big endian int 2019-04-05 17:30:35 -04:00
Stuckinaboot 3c0fca8979 Remove prints 2019-04-04 17:10:41 -04:00
Stuckinaboot 8221e1057c Merge branch 'seqno' of https://github.com/libp2p/py-libp2p into seqno 2019-04-04 17:09:51 -04:00
Stuckinaboot 22e503a260 Add test for ring topology multiple messages from two origins 2019-04-03 23:32:03 -04:00
Stuckinaboot fc78928037 Add test for multiple messages from two origins 2019-04-03 23:32:03 -04:00
Stuckinaboot 211d6f6860 Add dummy node test 2019-04-03 23:32:03 -04:00
Stuckinaboot feaa393c5f Fix seen messages bug 2019-04-03 23:32:03 -04:00
Stuckinaboot 9d16aa834d Modify pubsub to have seen message check incorporate seqno and node id 2019-04-03 23:32:03 -04:00
Stuckinaboot c2b5389362 Adjust floodsub tests for new seqno util 2019-04-03 23:32:03 -04:00
Stuckinaboot f6299c7dee Add priority queues to handle seqno 2019-04-03 23:32:03 -04:00
Stuckinaboot e62b532c9c Merge branch 'seqno' of https://github.com/libp2p/py-libp2p into HEAD 2019-04-03 15:16:24 -04:00
Stuckinaboot 0fe21308ae Add test for ring topology multiple messages from two origins 2019-04-03 15:13:57 -04:00
Stuckinaboot c270bda456 Add test for multiple messages from two origins 2019-04-03 15:13:57 -04:00
Stuckinaboot 3f2561e30b Add dummy node test 2019-04-03 15:13:57 -04:00
Stuckinaboot b73b2f0841 Fix seen messages bug 2019-04-03 15:13:57 -04:00
Stuckinaboot 91dd3c0230 Modify pubsub to have seen message check incorporate seqno and node id 2019-04-03 15:13:57 -04:00
Stuckinaboot a47b02dd26 Adjust floodsub tests for new seqno util 2019-04-03 15:13:56 -04:00
Stuckinaboot 502bb01242 Add priority queues to handle seqno 2019-04-03 15:13:56 -04:00
zixuanzh 7d3a8b5e77 clean up 2019-04-03 15:13:56 -04:00
zixuanzh bc6a27a762 fix all tests 2019-04-03 15:13:56 -04:00
zixuanzh 372ed8336c remove message from test dummy 2019-04-03 15:13:56 -04:00
zixuanzh 73c1d87db1 update dummy account node 2019-04-03 15:13:56 -04:00
zixuanzh fb5b3e4a24 reworked floodsub logic 2019-04-03 15:13:56 -04:00
zixuanzh 3836aa65f1 fix encoding issue 2019-04-03 15:13:56 -04:00
zixuanzh 65526a3319 remove Message from dummy account 2019-04-03 15:13:56 -04:00
zixuanzh 35a587f72a remove message.py 2019-04-03 15:13:56 -04:00
zixuanzh c03ba881f2 reworked subscribe unsubsrcibe 2019-04-03 15:13:56 -04:00
Alex Haynes 83de4f2972 RPC conversion progress 2019-04-03 15:13:56 -04:00
zixuanzh dea31531da update dependencies 2019-04-03 15:13:56 -04:00
zixuanzh a25aa0b034 rewrote get_hello_packet 2019-04-03 15:13:56 -04:00
zixuanzh f1a20f12dc update from to from_id in proto 2019-04-03 15:13:56 -04:00
zixuanzh 2aa24cbfe8 add generated rpc code 2019-04-03 15:13:56 -04:00
zixuanzh f655148d9c add pubsub proto 2019-04-03 15:13:56 -04:00
Alex Haynes 0d7b8c3e5d added rpc.proto from go repo 2019-04-03 15:13:56 -04:00
stuckinaboot 9c6d441f9f [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-04-03 15:13:56 -04:00
zixuanzh fee905ace2 fix linting issues 2019-04-03 15:13:56 -04:00
zixuanzh 2e437e5b8b add test for listen event 2019-04-03 15:13:56 -04:00
zixuanzh 2cb0279b57 fix existing tests 2019-04-03 15:13:56 -04:00
zixuanzh 4c89a59406 refactor host setup helper 2019-04-03 15:13:56 -04:00
zixuanzh fca5782d01 refactor new_node 2019-04-03 15:13:56 -04:00
Stuckinaboot ed3c5f4b51 Add additional initiator and non-initiator notifee tests 2019-04-03 15:13:56 -04:00
Stuckinaboot dfe2b0e7e7 Add return value to Notify 2019-04-03 15:13:56 -04:00
Stuckinaboot 4b9327c5a3 Add opened_stream call for non-initiator 2019-04-03 15:13:56 -04:00
Stuckinaboot a0ef64fff7 Fix outstanding unrelated lint issue in multiselect_client 2019-04-03 15:13:56 -04:00
Stuckinaboot 92bfd816b9 Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot 5cabc11687 Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot f58afc0eac Fix linting issues 2019-04-03 15:13:56 -04:00
Stuckinaboot fa97b97748 Refactor test setup to remove duplicate code 2019-04-03 15:13:56 -04:00
Stuckinaboot 86ce7530d5 Add conn to net stream and conn tests 2019-04-03 15:13:56 -04:00
Stuckinaboot 66703bd414 Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot 4298b0631f Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot 1ecdcf7df4 Make notifee functions all async 2019-04-03 15:13:56 -04:00
Stuckinaboot 3beec9d4ae Add tests for notify 2019-04-03 15:13:56 -04:00
Stuckinaboot c465141331 Implement notify feature 2019-04-03 15:13:56 -04:00
Stuckinaboot 1701fba42e Add notify function to network interface 2019-04-03 15:13:56 -04:00
Stuckinaboot 204c103d92 Add notifee interface 2019-04-03 15:13:56 -04:00
Stuckinaboot 005ea67e82 Merge branch 'seqno' of https://github.com/libp2p/py-libp2p into seqno 2019-04-03 14:45:51 -04:00
Stuckinaboot 0713cddbd1 Add test for ring topology multiple messages from two origins 2019-04-03 14:24:40 -04:00
Stuckinaboot db53976a6a Add test for multiple messages from two origins 2019-04-03 14:24:40 -04:00
Stuckinaboot 62d1bff888 Add dummy node test 2019-04-03 14:24:40 -04:00
Stuckinaboot 60fe42bf60 Fix seen messages bug 2019-04-03 14:24:40 -04:00
Stuckinaboot 547e7b3546 Modify pubsub to have seen message check incorporate seqno and node id 2019-04-03 14:24:40 -04:00
Stuckinaboot 2934f93fc9 Adjust floodsub tests for new seqno util 2019-04-03 14:24:40 -04:00
Stuckinaboot 56fbdf3f2f Add priority queues to handle seqno 2019-04-03 14:24:40 -04:00
zixuanzh 0de028a236 clean up 2019-04-03 14:24:40 -04:00
zixuanzh f50ba04178 fix all tests 2019-04-03 14:24:40 -04:00
zixuanzh f287c73236 remove message from test dummy 2019-04-03 14:24:40 -04:00
zixuanzh 974cb0b06f update dummy account node 2019-04-03 14:24:40 -04:00
zixuanzh 87269a9524 reworked floodsub logic 2019-04-03 14:24:40 -04:00
zixuanzh 1cbd909dc2 fix encoding issue 2019-04-03 14:24:40 -04:00
zixuanzh a2e22159b2 remove Message from dummy account 2019-04-03 14:24:40 -04:00
zixuanzh 2183bada41 remove message.py 2019-04-03 14:24:40 -04:00
zixuanzh 598b6b257f reworked subscribe unsubsrcibe 2019-04-03 14:24:40 -04:00
Alex Haynes 75c12d4aed RPC conversion progress 2019-04-03 14:24:40 -04:00
zixuanzh adce7f0a05 update dependencies 2019-04-03 14:24:40 -04:00
zixuanzh b297aa9576 rewrote get_hello_packet 2019-04-03 14:24:40 -04:00
zixuanzh 2b41818561 update from to from_id in proto 2019-04-03 14:24:40 -04:00
zixuanzh f2de3735ee add generated rpc code 2019-04-03 14:24:40 -04:00
zixuanzh 2aa7c4a106 add pubsub proto 2019-04-03 14:24:40 -04:00
Alex Haynes 1c08e9e55c added rpc.proto from go repo 2019-04-03 14:24:39 -04:00
stuckinaboot 8d0f40a378 [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-04-03 14:24:39 -04:00
zixuanzh fa53c5a866 fix linting issues 2019-04-03 14:24:39 -04:00
zixuanzh cc7c08bac4 add test for listen event 2019-04-03 14:24:39 -04:00
zixuanzh 3e0d6ea126 fix existing tests 2019-04-03 14:24:39 -04:00
zixuanzh 9aa41b106a refactor host setup helper 2019-04-03 14:24:39 -04:00
zixuanzh 474602c4f7 refactor new_node 2019-04-03 14:24:39 -04:00
Stuckinaboot 51cd9c8e56 Add additional initiator and non-initiator notifee tests 2019-04-03 14:24:39 -04:00
Stuckinaboot 7d45131f37 Add return value to Notify 2019-04-03 14:24:39 -04:00
Stuckinaboot d800b3ef41 Add opened_stream call for non-initiator 2019-04-03 14:24:39 -04:00
Stuckinaboot 1384cebf9c Fix outstanding unrelated lint issue in multiselect_client 2019-04-03 14:24:39 -04:00
Stuckinaboot ce22f06773 Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot ef827f90ad Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot 15883fcf09 Fix linting issues 2019-04-03 14:24:39 -04:00
Stuckinaboot b340b89819 Refactor test setup to remove duplicate code 2019-04-03 14:24:39 -04:00
Stuckinaboot 91b4d66d50 Add conn to net stream and conn tests 2019-04-03 14:24:39 -04:00
Stuckinaboot 86b4bb5e55 Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot a1c0165484 Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot 0d2aa3f5b1 Make notifee functions all async 2019-04-03 14:24:39 -04:00
Stuckinaboot 5a4af0c81c Add tests for notify 2019-04-03 14:24:39 -04:00
Stuckinaboot 3dbb969b0b Implement notify feature 2019-04-03 14:24:39 -04:00
Stuckinaboot 95f1f0bbf0 Add notify function to network interface 2019-04-03 14:24:39 -04:00
Stuckinaboot 510a5eaa34 Add notifee interface 2019-04-03 14:24:39 -04:00
Stuckinaboot c61f4297b0 Add test for ring topology multiple messages from two origins 2019-04-03 14:20:50 -04:00
Stuckinaboot 0ff155660d Add test for multiple messages from two origins 2019-04-03 14:17:33 -04:00
ZX 0190481e10
Merge pull request #141 from libp2p/floodsub-rpc
Floodsub RPC
2019-04-03 14:15:32 -04:00
zixuanzh 225bd390df add source to rpc.proto 2019-04-03 14:08:05 -04:00
zixuanzh 3a52d29cb7 remove redundant proto file 2019-04-03 14:05:37 -04:00
David Sanders 3c1bbfe738
Merge pull request #24 from davesque/updates
Update default author email
2019-04-03 11:55:57 -06:00
Stuckinaboot e6605f22e9 Add dummy node test 2019-04-03 00:35:39 -04:00
Stuckinaboot 583b79fad7 Fix seen messages bug 2019-04-03 00:34:39 -04:00
Stuckinaboot 726d083a3a Modify pubsub to have seen message check incorporate seqno and node id 2019-04-02 22:34:01 -04:00
Stuckinaboot 1296589800 Adjust floodsub tests for new seqno util 2019-04-02 22:05:32 -04:00
Stuckinaboot 71282678c4 Add priority queues to handle seqno 2019-04-02 22:05:14 -04:00
zixuanzh 0238dff217 remove unused code 2019-04-02 21:17:48 -04:00
zixuanzh 41d1aae55b clean up 2019-04-01 16:55:44 -04:00
zixuanzh 6eb070b78e fix all tests 2019-04-01 16:23:20 -04:00
zixuanzh 2e5e7e3c10 remove message from test dummy 2019-04-01 15:04:20 -04:00
zixuanzh de6bc011f0 update dummy account node 2019-03-31 22:16:49 -04:00
zixuanzh 89a19a9213 reworked floodsub logic 2019-03-31 22:16:28 -04:00
zixuanzh 971dbe1a96 fix encoding issue 2019-03-30 19:30:58 -04:00
zixuanzh ec7bc45a58 remove Message from dummy account 2019-03-30 19:12:31 -04:00
zixuanzh f5af4b9016 remove message.py 2019-03-30 18:49:50 -04:00
zixuanzh aec783b843 reworked subscribe unsubsrcibe 2019-03-30 17:59:08 -04:00
Alex Haynes bf17f424b3 RPC conversion progress 2019-03-29 16:23:30 -04:00
Alex Haynes 9be1214c44 Merge branch 'floodsub-rpc' of https://github.com/libp2p/py-libp2p into floodsub-rpc 2019-03-29 15:02:15 -04:00
zixuanzh a81628f99d update dependencies 2019-03-28 15:42:55 -04:00
zixuanzh 3cfeccaf17 rewrote get_hello_packet 2019-03-28 15:25:33 -04:00
zixuanzh 81d121a029 update from to from_id in proto 2019-03-28 15:23:56 -04:00
zixuanzh 1e8d93fcf6 add generated rpc code 2019-03-28 09:55:14 -04:00
zixuanzh 041e0fbe34 add pubsub proto 2019-03-28 09:41:38 -04:00
Christophe de Carvalho c5289952ee
add routing interfaces 2019-03-26 20:44:01 +01:00
Christophe de Carvalho 2d1b9a03d1
clean up 2019-03-26 20:20:14 +01:00
Christophe de Carvalho cd8cb5c443
refactoring of the code to implement IAdvertiser and IDiscoverer 2019-03-26 20:16:16 +01:00
Christophe de Carvalho Pereira Martins 4889a0a790
First POC of peer routing using kademlia lib 2019-03-26 20:15:19 +01:00
Christophe de Carvalho Pereira Martins c24a279d2d
update kadmelia lib 2019-03-26 20:14:42 +01:00
David Sanders 5e6dc29991
Update default author email 2019-03-26 12:11:50 -06:00
David Sanders 0b85056ca8
Merge pull request #23 from davesque/travis
Remove travis config
2019-03-26 11:49:07 -06:00
David Sanders 5f3e6b37bf
Remove travis config 2019-03-25 20:04:26 -06:00
David Sanders 7bffe4b08c
Merge pull request #22 from davesque/updates
Remove attribution to specific person, update README
2019-03-25 17:52:00 -06:00
David Sanders de4bdf9e30
Remove attribution to specific person 2019-03-25 15:14:04 -06:00
Alex Haynes cb6b76d3b5 added rpc.proto from go repo 2019-03-24 15:08:36 -04:00
stuckinaboot 57077cd3b4 [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-03-23 13:52:02 -04:00
ZX 6c4bbd1e85
Merge pull request #139 from libp2p/swarm-refactor
Refactor libp2p new_node creation
2019-03-19 00:03:24 -04:00
zixuanzh d67aeb1974 fix linting issues 2019-03-17 21:30:56 -04:00
zixuanzh 0d15126ffc add test for listen event 2019-03-17 21:15:14 -04:00
zixuanzh 1886258fbf fix existing tests 2019-03-17 19:33:40 -04:00
zixuanzh 72a8d55faa refactor host setup helper 2019-03-17 19:33:10 -04:00
zixuanzh fea26d9087 refactor new_node 2019-03-17 19:20:46 -04:00
Stuckinaboot d42a0d7e9f Add additional initiator and non-initiator notifee tests 2019-03-15 11:22:20 -04:00
Stuckinaboot a5fa053815 Add return value to Notify 2019-03-15 11:22:20 -04:00
Stuckinaboot 01099e9f2c Add opened_stream call for non-initiator 2019-03-15 11:22:20 -04:00
Stuckinaboot 745c5e677c Fix outstanding unrelated lint issue in multiselect_client 2019-03-15 11:22:20 -04:00
Stuckinaboot 58bdee499b Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot a46779918f Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot 0c60e28150 Fix linting issues 2019-03-15 11:22:20 -04:00
Stuckinaboot 2eb8382280 Refactor test setup to remove duplicate code 2019-03-15 11:22:20 -04:00
Stuckinaboot 22d6c97ef3 Add conn to net stream and conn tests 2019-03-15 11:22:20 -04:00
Stuckinaboot 26dd385c95 Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot 670076035c Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot 3e64520506 Make notifee functions all async 2019-03-15 11:22:20 -04:00
Stuckinaboot 7a8878411f Add tests for notify 2019-03-15 11:22:20 -04:00
Stuckinaboot 1ce8225c7d Implement notify feature 2019-03-15 11:22:20 -04:00
Stuckinaboot f7d1bb3a89 Add notify function to network interface 2019-03-15 11:22:20 -04:00
Stuckinaboot cdfcdd528e Add notifee interface 2019-03-15 11:22:20 -04:00
Hyungsuk Kang 78d0a658bb Typo fixes (#127)
* Update transport_interface.py

* Update tcp.py

* Update README.md
2019-02-24 23:11:54 -05:00
Robert Zajac 82840b5e6c
Stream rearchitecture (#126)
* Add generic protocol handler

* Add generic protocol handler to stream muxing pipeline

* Modify conn_handler to only deal with connections

* mplex accept stream architecture changes

* Add create generic protocol handler

* Fix minor bugs

* who would win 4 devs or one not

* Debugging

* rearch with handle_incoming infinite loop, seems to work, needs cleanup"

* passing linting, still needs cleanup

* fixing linting again; code still needs cleanup

* fixing tests; code still needs cleanup

* adding test cleanup and task cleanup, removing prints

* linting, and cleanup complete

* storing connections based on peer id

* remove dead code

* remove unnecessary peer_id
2019-02-24 20:58:23 -05:00
Alex Haynes 17c778de15
Peer Discovery Interface (#123)
* added file

* basic interface modeled on go repo

* fixed linting

* updated based on comments
2019-02-24 18:37:56 -05:00
Robert Zajac e3e3ac30b1
Adding sponsorship ack to EF (#125)
Now that sponsorship has been announced (see link in blog), we can add acknowledgement and close out #60.
2019-02-22 15:02:17 -05:00
Alex Haynes 934714df62 updated readme (#124) 2019-02-12 10:56:03 -05:00
Robert Zajac e34603c364
fixing chat to use public IP (#122) 2019-02-10 20:52:05 -05:00
David Sanders 2fb32d204d
Merge pull request #21 from davesque/pydocstyle
Add docstring checking with pydocstyle
2019-01-25 10:00:24 -07:00
David Sanders dd74824840
Add docstring checking with pydocstyle 2019-01-24 13:57:17 -07:00
Jason Carver 66e6b7e3c4
Merge pull request #19 from ethereum/drop-py35
Drop py3.5, add py3.7
2019-01-16 13:55:56 -08:00
Jason Carver caf9050a19 Drop py3.5, add py3.7 2019-01-15 16:06:18 -08:00
Jason Carver 03f00d3fbe
Merge pull request #12 from njgheorghita/update-bumpversion
[WIP] Add changelog instructions
2018-09-05 11:00:43 -07:00
Nick Gheorghita 0b004a8597
Add changelog instructions 2018-09-04 18:46:05 -06:00
Jason Carver 5ad0973c3d
Ignore any internal `_utils` module 2018-08-29 09:47:22 -07:00
Jason Carver 56e4b20080
Merge pull request #7 from davesque/fix-readlink
Make template filler more BSD friendly
2018-06-07 12:04:01 -07:00
Jason Carver af727b97f2
Merge pull request #8 from davesque/twine
Use twine for pypi uploading per packaging docs
2018-06-07 12:02:28 -07:00
Jason Carver e0af03a299
Merge pull request #10 from davesque/eth-utils-version
Update eth-utils version requirement
2018-06-07 11:52:59 -07:00
Jason Carver 24d6899caa
Merge pull request #9 from davesque/gpg-sign-setting
Fix ineffectual commands in Makefile
2018-06-07 11:52:30 -07:00
David Sanders d3537cf1ab
Update eth-utils version requirement 2018-06-07 12:27:07 -06:00
David Sanders a9d531615d
Fix ineffectual commands in Makefile 2018-06-07 12:23:32 -06:00
David Sanders 016f48d17d
Use twine for pypi uploading per packaging docs
See here:
https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives
2018-06-06 19:35:31 -06:00
David Sanders 6ed7bb011f
Make template filler more BSD friendly 2018-06-06 18:49:12 -06:00
Jason Carver 83ce66097c
Merge pull request #6 from davesque/isort-config
Fix possibly incorrect isort config
2018-06-06 11:32:12 -07:00
David Sanders 0088da3781
Fix possibly incorrect isort config 2018-06-04 18:40:58 -06:00
Jason Carver 28787fe9f6
Merge pull request #5 from davesque/misc-fixes
Misc fixes
2018-06-04 16:45:32 -06:00
David Sanders 0a83658b3e
Lint in setup.py 2018-06-04 16:37:52 -06:00
David Sanders e7d0a8577b
Fix testing dependencies 2018-06-04 16:37:38 -06:00
David Sanders 00819dd924
Whitespace 2018-06-04 16:36:35 -06:00
David Sanders 32c8c981f2
Remove references to deprecated "py.test" 2018-06-04 16:36:02 -06:00
Jason Carver 8352ad5afa
Merge pull request #4 from davesque/readme-updates
Some readme updates
2018-06-01 14:14:56 -06:00
David Sanders 5e509100e7
Some readme updates 2018-06-01 14:05:57 -06:00
Jason Carver ac85506912
make merge script executable 2018-05-15 13:02:56 -07:00
Jason Carver b357fda082 refill from any directory 2018-05-15 12:33:36 -07:00
Jason Carver 71ba221c49 Enable template refilling 2018-05-15 12:29:21 -07:00
Jason Carver 85e33cf380 move template filler to own directory 2018-05-15 12:17:36 -07:00
Jason Carver 1091971637 remove old merge code from circle template 2018-05-15 12:09:51 -07:00
Jason Carver c18b0cecfc
Merge pull request #3 from carver/circle-merge
circleci: merge (and reattempt) PR before testing
2018-05-15 12:02:29 -07:00
Jason Carver 8815766abf circleci: merge (and reattempt) PR before testing 2018-05-15 11:54:33 -07:00
Jason Carver 1f89c6385a fix gitignore typo about internals module in docs/ 2018-04-25 18:50:01 -07:00
Jason Carver 285f5c6c70 Remove duplicate python version mention in readme
Rely on the badge instead
2018-04-25 15:39:52 -07:00
Jason Carver b98147efca keep template filler around for future merges 2018-04-25 15:20:03 -07:00
Jason Carver 0dd03f0a3f re-add MODULE_NAME in new template filler 2018-04-25 15:19:37 -07:00
Jason Carver d7ce1100ed gitignore utils and internals docs; fill-in fixup 2018-04-25 15:05:30 -07:00
Jason Carver 5a9574a72c
Merge pull request #2 from davesque/updates
Couple more updates
2018-04-23 14:08:04 -07:00
David Sanders cdc83bd498
Make find replace script more macOS friendly
Also, make handling of arbitrary filenames a bit more robust.
2018-04-22 01:35:08 -06:00
David Sanders f7535eb992
Add circle ci status badge 2018-04-22 01:34:42 -06:00
Jason Carver 43960ffb5e
Merge pull request #1 from davesque/circleci
Make Circle CI test against merge with PR base
2018-04-19 16:16:38 -07:00
David Sanders 30473b3ff5
Make Circle CI test against merge with PR base 2018-04-19 16:31:48 -06:00
Jason Carver 84ecb7effc require python 3.5 during setup (and reject py4) 2018-04-10 10:16:17 -07:00
Jason Carver 031b1175f4 force tox to rebuild in circle runs 2018-02-28 14:20:39 -08:00
Jason Carver 588b1af6ee Makefile: deploy to upstream 2018-02-28 10:29:45 -08:00
Jason Carver 974fae3a2b travis config: use latest pypy3 container 2018-02-20 18:10:30 -08:00
Jason Carver f0e0b10cc5 add pypy3 as a default test environment in circle 2018-02-20 18:09:58 -08:00
Jason Carver b3461e9c93 tox.ini bugfix: whitelist make 2018-02-07 14:40:01 -08:00
Jason Carver 58902032c7 add doctest run to circle-ci 2018-02-07 14:37:30 -08:00
Jason Carver 60b026e3ed add doctest run to travis 2018-02-07 11:40:35 -08:00
Jason Carver ecb3731cbe can't override python that way in tox 2018-02-07 11:26:47 -08:00
Jason Carver 37d978c5b3 squash warning about using make inside tox 2018-02-07 11:24:37 -08:00
Jason Carver a1ba89ed41 every tox environment should have a basic python 2018-02-07 11:18:47 -08:00
Jason Carver a9d9fec258 add doctest by default 2018-02-07 11:14:11 -08:00
Jason Carver beb1b10ee8 silence flake8 error during module import test 2018-02-06 16:10:25 -08:00
Jason Carver 1695a3b326 bugfix: setuptools reference to pypy 2018-02-06 16:04:41 -08:00
Jason Carver 6abf066779 add module import test 2018-02-06 15:56:11 -08:00
Jason Carver 7bfa2ac5d4 add pypy3 as a supported environment, by default 2018-02-06 15:55:49 -08:00
Jason Carver b825c36def docs: show badge, link to release notes 2018-02-06 15:42:21 -08:00
Jason Carver 4bb4f720db add badges: pypi version and pythons supported 2018-02-06 15:33:04 -08:00
Jason Carver f7b0f07b45 fixups: rm template filler, xfail strict, make bug 2018-02-01 13:07:33 -08:00
Jason Carver cb71fb4430 add circle ci config 2018-02-01 09:42:48 -08:00
Jason Carver 9a7d72a816 pytest default settings, plus pytest-watch 2018-01-30 15:38:42 -08:00
Jason Carver 0e57d8e262 when do you not want ipython in local dev? never 2018-01-30 15:18:59 -08:00
Jason Carver 96a371705b bugfix: run tox -elint *after* isort
otherwise tox fails, and the auto-lint never runs
2018-01-30 15:17:50 -08:00
Jason Carver 109a0866ed requirements.txt: best way to build on readthedocs 2018-01-30 15:17:03 -08:00
Jason Carver 195249cdab docs cleanup: Makefile, conf, warnings 2018-01-30 11:18:16 -08:00
Jason Carver 76bfefa66c add `make lint-roll` to locally auto de-lint 2018-01-26 19:01:36 -08:00
Jason Carver 2938497f02 Link to dev tactical manual 2018-01-25 17:21:42 -08:00
Jason Carver 3290919ff1 add github templates for issues and pull requests 2018-01-25 15:51:14 -08:00
Jason Carver 1d4e19d929 add docs index 2018-01-24 17:07:48 -08:00
Jason Carver ca670f43a4 start at devnum 0 so the first bump goes to 1 2018-01-24 16:35:36 -08:00
Jason Carver 4a8b7b06af shorten extra name for documentation install 2018-01-24 16:33:03 -08:00
Jason Carver 9cdfd649d5 create empty module with supplied name 2018-01-24 16:31:02 -08:00
Jason Carver cdba56a295 skip replacing variables inside .git/* files 2018-01-24 16:24:31 -08:00
Jason Carver 3d87ff7153 Replace all template vars 2018-01-24 16:24:08 -08:00
Jason Carver a7955a560e init 2018-01-24 16:00:28 -08:00
270 changed files with 18060 additions and 3251 deletions

23
.bumpversion.cfg Normal file
View File

@ -0,0 +1,23 @@
[bumpversion]
current_version = 0.1.5
commit = True
tag = True
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(-(?P<stage>[^.]*)\.(?P<devnum>\d+))?
serialize =
{major}.{minor}.{patch}-{stage}.{devnum}
{major}.{minor}.{patch}
[bumpversion:part:stage]
optional_value = stable
first_value = stable
values =
alpha
beta
stable
[bumpversion:part:devnum]
[bumpversion:file:setup.py]
search = version="{current_version}",
replace = version="{new_version}",

77
.circleci/config.yml Normal file
View File

@ -0,0 +1,77 @@
version: 2.0
# heavily inspired by https://raw.githubusercontent.com/pinax/pinax-wiki/6bd2a99ab6f702e300d708532a6d1d9aa638b9f8/.circleci/config.yml
common: &common
working_directory: ~/repo
steps:
- checkout
- run:
name: merge pull request base
command: ./.circleci/merge_pr.sh
- run:
name: merge pull request base (2nd try)
command: ./.circleci/merge_pr.sh
when: on_fail
- run:
name: merge pull request base (3nd try)
command: ./.circleci/merge_pr.sh
when: on_fail
- restore_cache:
keys:
- cache-{{ .Environment.CIRCLE_JOB }}-{{ checksum "setup.py" }}-{{ checksum "tox.ini" }}
- run:
name: install dependencies
command: pip install --user tox
- run:
name: run tox
command: ~/.local/bin/tox -r
- save_cache:
paths:
- .hypothesis
- .tox
- ~/.cache/pip
- ~/.local
- ./eggs
key: cache-{{ .Environment.CIRCLE_JOB }}-{{ checksum "setup.py" }}-{{ checksum "tox.ini" }}
jobs:
docs:
<<: *common
docker:
- image: circleci/python:3.6
environment:
TOXENV: docs
lint:
<<: *common
docker:
- image: circleci/python:3.6
environment:
TOXENV: lint
py36-core:
<<: *common
docker:
- image: circleci/python:3.6
environment:
TOXENV: py36-core
py37-core:
<<: *common
docker:
- image: circleci/python:3.7
environment:
TOXENV: py37-core
pypy3-core:
<<: *common
docker:
- image: pypy
environment:
TOXENV: pypy3-core
workflows:
version: 2
test:
jobs:
- docs
- lint
- py36-core
- py37-core
- pypy3-core

12
.circleci/merge_pr.sh Executable file
View File

@ -0,0 +1,12 @@
#!/usr/bin/env bash
if [[ -n "${CIRCLE_PR_NUMBER}" ]]; then
PR_INFO_URL=https://api.github.com/repos/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/pulls/$CIRCLE_PR_NUMBER
PR_BASE_BRANCH=$(curl -L "$PR_INFO_URL" | python -c 'import json, sys; obj = json.load(sys.stdin); sys.stdout.write(obj["base"]["ref"])')
git fetch origin +"$PR_BASE_BRANCH":circleci/pr-base
# We need these config values or git complains when creating the
# merge commit
git config --global user.name "Circle CI"
git config --global user.email "circleci@example.com"
git merge --no-edit circleci/pr-base
fi

38
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,38 @@
_If this is a bug report, please fill in the following sections.
If this is a feature request, delete and describe what you would like with examples._
## What was wrong?
### Code that produced the error
```py
CODE_TO_REPRODUCE
```
### Full error output
```sh
ERROR_HERE
```
### Expected Result
_This section may be deleted if the expectation is "don't crash"._
```sh
EXPECTED_RESULT
```
### Environment
```sh
# run this:
$ python -m eth_utils
# then copy the output here:
OUTPUT_HERE
```
## How can it be fixed?
Fill this section in if you know how this could or should be fixed.

21
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,21 @@
## What was wrong?
Issue #
## How was it fixed?
Summary of approach.
### To-Do
[//]: # (Stay ahead of things, add list items here!)
- [ ] Clean up commit history
[//]: # (For important changes that should go into the release notes please add a newsfragment file as explained here: https://github.com/libp2p/py-libp2p/blob/master/newsfragments/README.md)
[//]: # (See: https://py-libp2p.readthedocs.io/en/latest/contributing.html#pull-requests)
- [ ] Add entry to the [release notes](https://github.com/libp2p/py-libp2p/blob/master/newsfragments/README.md)
#### Cute Animal Picture
![put a cute animal picture link inside the parentheses]()

143
.gitignore vendored
View File

@ -1,56 +1,133 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
__pycache__/
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
*.egg-info
dist
build
eggs
.eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
venv*
.Python
downloads/
wheels/
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
pip-wheel-metadata
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
.tox
nosetests.xml
htmlcov/
.coverage.*
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
# Complexity
output/*.html
output/*/index.html
# Sphinx
docs/_build
docs/modules.rst
docs/*.internal.rst
docs/*._utils.*
# Hypothese Property base testing
.hypothesis
# tox/pytest cache
.cache
# Test output logs
logs
### JetBrains template
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
# User-specific stuff:
.idea/workspace.xml
.idea/tasks.xml
.idea/dictionaries
.idea/vcs.xml
.idea/jsLibraryMappings.xml
# Sensitive or high-churn files:
.idea/dataSources.ids
.idea/dataSources.xml
.idea/dataSources.local.xml
.idea/sqlDataSources.xml
.idea/dynamic.xml
.idea/uiDesigner.xml
# Gradle:
.idea/gradle.xml
.idea/libraries
# Mongo Explorer plugin:
.idea/mongoSettings.xml
# VIM temp files
*.sw[op]
# mypy
.mypy_cache
## File-based project format:
*.iws
## Plugin-specific files:
# IntelliJ
/out/
# mpeltonen/sbt-idea plugin
.idea_modules/
# JIRA plugin
atlassian-ide-plugin.xml
# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Django stuff:
*.log
local_settings.py
@ -63,9 +140,6 @@ instance/
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
@ -85,10 +159,8 @@ celerybeat-schedule
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
@ -100,8 +172,5 @@ venv.bak/
# mkdocs documentation
/site
# mypy
.mypy_cache/
# pycharm
.idea/
# vscode
.vscode/

View File

@ -0,0 +1,48 @@
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
PROJECT_ROOT=$(dirname $(dirname $(python -c 'import os, sys; sys.stdout.write(os.path.realpath(sys.argv[1]))' "$0")))
echo "What is your python module name?"
read MODULE_NAME
echo "What is your pypi package name? (default: $MODULE_NAME)"
read PYPI_INPUT
PYPI_NAME=${PYPI_INPUT:-$MODULE_NAME}
echo "What is your github project name? (default: $PYPI_NAME)"
read REPO_INPUT
REPO_NAME=${REPO_INPUT:-$PYPI_NAME}
echo "What is your readthedocs.org project name? (default: $PYPI_NAME)"
read RTD_INPUT
RTD_NAME=${RTD_INPUT:-$PYPI_NAME}
echo "What is your project name (ex: at the top of the README)? (default: $REPO_NAME)"
read PROJECT_INPUT
PROJECT_NAME=${PROJECT_INPUT:-$REPO_NAME}
echo "What is a one-liner describing the project?"
read SHORT_DESCRIPTION
_replace() {
local find_cmd=(find "$PROJECT_ROOT" ! -perm -u=x ! -path '*/.git/*' -type f)
if [[ $(uname) == Darwin ]]; then
"${find_cmd[@]}" -exec sed -i '' "$1" {} +
else
"${find_cmd[@]}" -exec sed -i "$1" {} +
fi
}
_replace "s/<MODULE_NAME>/$MODULE_NAME/g"
_replace "s/<PYPI_NAME>/$PYPI_NAME/g"
_replace "s/<REPO_NAME>/$REPO_NAME/g"
_replace "s/<RTD_NAME>/$RTD_NAME/g"
_replace "s/<PROJECT_NAME>/$PROJECT_NAME/g"
_replace "s/<SHORT_DESCRIPTION>/$SHORT_DESCRIPTION/g"
mkdir -p "$PROJECT_ROOT/$MODULE_NAME"
touch "$PROJECT_ROOT/$MODULE_NAME/__init__.py"

View File

@ -0,0 +1,2 @@
TEMPLATE_DIR=$(dirname $(readlink -f "$0"))
<"$TEMPLATE_DIR/template_vars.txt" "$TEMPLATE_DIR/fill_template_vars.sh"

View File

@ -0,0 +1,6 @@
libp2p
libp2p
py-libp2p
py-libp2p
py-libp2p
The Python implementation of the libp2p networking stack

30
.pydocstyle.ini Normal file
View File

@ -0,0 +1,30 @@
[pydocstyle]
; All error codes found here:
; http://www.pydocstyle.org/en/3.0.0/error_codes.html
;
; Ignored:
; D1 - Missing docstring error codes
;
; Selected:
; D2 - Whitespace error codes
; D3 - Quote error codes
; D4 - Content related error codes
select=D2,D3,D4
; Extra ignores:
; D200 - One-line docstring should fit on one line with quotes
; D203 - 1 blank line required before class docstring
; D204 - 1 blank line required after class docstring
; D205 - 1 blank line required between summary line and description
; D212 - Multi-line docstring summary should start at the first line
; D302 - Use u""" for Unicode docstrings
; D400 - First line should end with a period
; D401 - First line should be in imperative mood
; D412 - No blank lines allowed between a section header and its content
add-ignore=D200,D203,D204,D205,D212,D302,D400,D401,D412
; Explanation:
; D400 - Enabling this error code seems to make it a requirement that the first
; sentence in a docstring is not split across two lines. It also makes it a
; requirement that no docstring can have a multi-sentence description without a
; summary line. Neither one of those requirements seem appropriate.

533
.pylintrc
View File

@ -1,533 +0,0 @@
[MASTER]
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS
# Add files or directories matching the regex patterns to the blacklist. The
# regex matches against base names, not paths.
ignore-patterns=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Use multiple processes to speed up Pylint.
jobs=1
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
# Pickle collected data for later comparisons.
persistent=yes
# Specify a configuration file.
rcfile=.pylintrc
# When enabled, pylint would attempt to guess common misconfiguration and emit
# user-friendly hints instead of false-positive error messages
suggestion-mode=yes
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=print-statement,
parameter-unpacking,
unpacking-in-except,
old-raise-syntax,
backtick,
long-suffix,
old-ne-operator,
old-octal-literal,
import-star-module-level,
non-ascii-bytes-literal,
raw-checker-failed,
bad-inline-option,
locally-disabled,
locally-enabled,
file-ignored,
suppressed-message,
useless-suppression,
deprecated-pragma,
apply-builtin,
basestring-builtin,
buffer-builtin,
cmp-builtin,
coerce-builtin,
execfile-builtin,
file-builtin,
long-builtin,
raw_input-builtin,
reduce-builtin,
standarderror-builtin,
unicode-builtin,
xrange-builtin,
coerce-method,
delslice-method,
getslice-method,
setslice-method,
no-absolute-import,
old-division,
dict-iter-method,
dict-view-method,
next-method-called,
metaclass-assignment,
indexing-exception,
raising-string,
reload-builtin,
oct-method,
hex-method,
nonzero-method,
cmp-method,
input-builtin,
round-builtin,
intern-builtin,
unichr-builtin,
map-builtin-not-iterating,
zip-builtin-not-iterating,
range-builtin-not-iterating,
filter-builtin-not-iterating,
using-cmp-argument,
eq-without-hash,
div-method,
idiv-method,
rdiv-method,
exception-message-attribute,
invalid-str-codec,
sys-max-int,
bad-python3-import,
deprecated-string-function,
deprecated-str-translate-call,
deprecated-itertools-function,
deprecated-types-field,
next-method-defined,
dict-items-not-iterating,
dict-keys-not-iterating,
dict-values-not-iterating,
missing-docstring,
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=c-extension-no-member
[REPORTS]
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
# Set the output format. Available formats are text, parseable, colorized, json
# and msvs (visual studio).You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
output-format=text
# Tells whether to display a full report or only the messages
reports=no
# Activate the evaluation score.
score=no
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[SPELLING]
# Limits count of emitted suggestions for spelling mistakes
max-spelling-suggestions=4
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,
XXX
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The minimum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,
_cb
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=six.moves,past.builtins,future.builtins
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=" "
# Maximum number of characters on a single line.
max-line-length=100
# Maximum number of lines in a module
max-module-lines=1000
# List of optional constructs for which whitespace checking is disabled. `dict-
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
# `empty-line` allows space-only lines.
no-space-check=trailing-comma,
dict-separator
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[SIMILARITIES]
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=yes
# Minimum lines number of a similarity.
min-similarity-lines=8
[BASIC]
# Naming style matching correct argument names
argument-naming-style=snake_case
# Regular expression matching correct argument names. Overrides argument-
# naming-style
#argument-rgx=
# Naming style matching correct attribute names
attr-naming-style=snake_case
# Regular expression matching correct attribute names. Overrides attr-naming-
# style
#attr-rgx=
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,
bar,
baz,
toto,
tutu,
tata
# Naming style matching correct class attribute names
class-attribute-naming-style=any
# Regular expression matching correct class attribute names. Overrides class-
# attribute-naming-style
#class-attribute-rgx=
# Naming style matching correct class names
class-naming-style=PascalCase
# Regular expression matching correct class names. Overrides class-naming-style
#class-rgx=
# Naming style matching correct constant names
const-naming-style=UPPER_CASE
# Regular expression matching correct constant names. Overrides const-naming-
# style
#const-rgx=
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming style matching correct function names
function-naming-style=snake_case
# Regular expression matching correct function names. Overrides function-
# naming-style
#function-rgx=
# Good variable names which should always be accepted, separated by a comma
good-names=i,
j,
k,
ex,
Run,
_
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Naming style matching correct inline iteration names
inlinevar-naming-style=any
# Regular expression matching correct inline iteration names. Overrides
# inlinevar-naming-style
#inlinevar-rgx=
# Naming style matching correct method names
method-naming-style=snake_case
# Regular expression matching correct method names. Overrides method-naming-
# style
#method-rgx=
# Naming style matching correct module names
module-naming-style=snake_case
# Regular expression matching correct module names. Overrides module-naming-
# style
#module-rgx=
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
property-classes=abc.abstractproperty
# Naming style matching correct variable names
variable-naming-style=snake_case
# Regular expression matching correct variable names. Overrides variable-
# naming-style
#variable-rgx=
[IMPORTS]
# Allow wildcard imports from modules that define __all__.
allow-wildcard-with-all=no
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=optparse,tkinter.tix
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
# Force import order to recognize a module as part of the standard
# compatibility libraries.
known-standard-library=
# Force import order to recognize a module as part of a third party library.
known-third-party=enchant
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,
__new__,
setUp
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,
_fields,
_replace,
_source,
_make
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# Maximum number of arguments for function / method
max-args=5
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of statements in function / method body
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=Exception

View File

@ -2,22 +2,36 @@ language: python
matrix:
include:
- python: 3.6-dev
dist: xenial
env: TOXENV=py36-test
- python: 3.7
dist: xenial
env: TOXENV=py37-test
- python: 3.7
dist: xenial
env: TOXENV=lint
- python: 3.7
dist: xenial
env: TOXENV=docs
- python: 3.7
dist: xenial
env: TOXENV=py37-interop GOBINPKG=go1.13.8.linux-amd64.tar.gz
sudo: true
before_install:
- wget https://dl.google.com/go/$GOBINPKG
- sudo tar -C /usr/local -xzf $GOBINPKG
- export GOPATH=$HOME/go
- export GOROOT=/usr/local/go
- export PATH=$GOROOT/bin:$GOPATH/bin:$PATH
- ./tests_interop/go_pkgs/install_interop_go_pkgs.sh
install:
- pip install --upgrade pip
- pip install -r requirements_dev.txt
- python setup.py develop
- pip install tox
script:
- pytest --cov=./libp2p tests/
- pylint --rcfile=.pylintrc libp2p/!(kademlia) tests
after_success:
- codecov
- tox
notifications:
slack: py-libp2p:RK0WVoQZhQXLgIKfHNPL1TR2

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2019 The Ethereum Foundation
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

113
Makefile Normal file
View File

@ -0,0 +1,113 @@
CURRENT_SIGN_SETTING := $(shell git config commit.gpgSign)
.PHONY: clean-pyc clean-build docs
help:
@echo "clean-build - remove build artifacts"
@echo "clean-pyc - remove Python file artifacts"
@echo "lint - check style with flake8, etc"
@echo "lint-roll - auto-correct styles with isort, black, docformatter, etc"
@echo "test - run tests quickly with the default Python"
@echo "testall - run tests on every Python version with tox"
@echo "release - package and upload a release"
@echo "dist - package"
FILES_TO_LINT = libp2p tests tests_interop examples setup.py
PB = libp2p/crypto/pb/crypto.proto \
libp2p/pubsub/pb/rpc.proto \
libp2p/security/insecure/pb/plaintext.proto \
libp2p/security/secio/pb/spipe.proto \
libp2p/security/noise/pb/noise.proto \
libp2p/identity/identify/pb/identify.proto
PY = $(PB:.proto=_pb2.py)
PYI = $(PB:.proto=_pb2.pyi)
# Set default to `protobufs`, otherwise `format` is called when typing only `make`
all: protobufs
protobufs: $(PY)
%_pb2.py: %.proto
protoc --python_out=. --mypy_out=. $<
clean-proto:
rm -f $(PY) $(PYI)
clean: clean-build clean-pyc
clean-build:
rm -fr build/
rm -fr dist/
rm -fr *.egg-info
clean-pyc:
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
find . -name '__pycache__' -exec rm -rf {} +
lint:
mypy -p libp2p -p examples --config-file mypy.ini
flake8 $(FILES_TO_LINT)
black --check $(FILES_TO_LINT)
isort --recursive --check-only --diff $(FILES_TO_LINT)
docformatter --pre-summary-newline --check --recursive $(FILES_TO_LINT)
tox -e lint # This is probably redundant, but just in case...
lint-roll:
isort --recursive $(FILES_TO_LINT)
black $(FILES_TO_LINT)
docformatter -ir --pre-summary-newline $(FILES_TO_LINT)
$(MAKE) lint
test:
pytest tests
test-all:
tox
build-docs:
sphinx-apidoc -o docs/ . setup.py "*conftest*" "libp2p/tools/interop*"
$(MAKE) -C docs clean
$(MAKE) -C docs html
$(MAKE) -C docs doctest
./newsfragments/validate_files.py
towncrier --draft --version preview
docs: build-docs
open docs/_build/html/index.html
linux-docs: build-docs
xdg-open docs/_build/html/index.html
package: clean
python setup.py sdist bdist_wheel
python scripts/release/test_package.py
notes:
# Let UPCOMING_VERSION be the version that is used for the current bump
$(eval UPCOMING_VERSION=$(shell bumpversion $(bump) --dry-run --list | grep new_version= | sed 's/new_version=//g'))
# Now generate the release notes to have them included in the release commit
towncrier --yes --version $(UPCOMING_VERSION)
# Before we bump the version, make sure that the towncrier-generated docs will build
make build-docs
git commit -m "Compile release notes"
release: clean
# require that you be on a branch that's linked to upstream/master
git status -s -b | head -1 | grep "\.\.upstream/master"
# verify that docs build correctly
./newsfragments/validate_files.py is-empty
make build-docs
CURRENT_SIGN_SETTING=$(git config commit.gpgSign)
git config commit.gpgSign true
bumpversion $(bump)
git push upstream && git push upstream --tags
python setup.py sdist bdist_wheel
twine upload dist/*
git config commit.gpgSign "$(CURRENT_SIGN_SETTING)"
dist: clean
python setup.py sdist bdist_wheel
ls -l dist

103
README.md
View File

@ -1,39 +1,86 @@
# py-libp2p [![Build Status](https://travis-ci.com/zixuanzh/py-libp2p.svg?branch=master)](https://travis-ci.com/zixuanzh/py-libp2p) [![codecov](https://codecov.io/gh/zixuanzh/py-libp2p/branch/master/graph/badge.svg)](https://codecov.io/gh/zixuanzh/py-libp2p) [![Gitter chat](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/py-libp2p/Lobby)[![Freenode](https://img.shields.io/badge/freenode-%23libp2p-yellow.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23libp2p)
# py-libp2p
[![Join the chat at https://gitter.im/py-libp2p/Lobby](https://badges.gitter.im/py-libp2p/Lobby.png)](https://gitter.im/py-libp2p/Lobby)
[![Build Status](https://travis-ci.com/libp2p/py-libp2p.svg?branch=master)](https://travis-ci.com/libp2p/py-libp2p)
[![PyPI version](https://badge.fury.io/py/libp2p.svg)](https://badge.fury.io/py/libp2p)
[![Python versions](https://img.shields.io/pypi/pyversions/libp2p.svg)](https://pypi.python.org/pypi/libp2p)
[![Docs build](https://readthedocs.org/projects/py-libp2p/badge/?version=latest)](http://py-libp2p.readthedocs.io/en/latest/?badge=latest)
[![Freenode](https://img.shields.io/badge/freenode-%23libp2p-yellow.svg)](https://webchat.freenode.net/?channels=%23libp2p)
[![Matrix](https://img.shields.io/badge/matrix-%23libp2p%3Apermaweb.io-blue.svg)](https://riot.permaweb.io/#/room/#libp2p:permaweb.io)
[![Discord](https://img.shields.io/discord/475789330380488707?color=blueviolet&label=discord)](https://discord.gg/66KBrm2)
<h1 align="center">
<img width="250" align="center" src="https://github.com/zixuanzh/py-libp2p/blob/master/assets/py-libp2p-logo.png?raw=true" alt="py-libp2p hex logo" />
<img width="250" align="center" src="https://github.com/libp2p/py-libp2p/blob/master/assets/py-libp2p-logo.png?raw=true" alt="py-libp2p hex logo" />
</h1>
## WARNING
py-libp2p is an experimental and work-in-progress repo under heavy development. We do not yet recommend using py-libp2p in production environments.
The Python implementation of the libp2p networking stack
Read more in the [documentation on ReadTheDocs](https://py-libp2p.readthedocs.io/). [View the release notes](https://py-libp2p.readthedocs.io/en/latest/release_notes.html).
## Sponsorship
This project is graciously sponsored by the Ethereum Foundation through [Wave 5 of their Grants Program](https://blog.ethereum.org/2019/02/21/ethereum-foundation-grants-program-wave-5/).
## Maintainers
The py-libp2p team consists of:
[@zixuanzh](https://github.com/zixuanzh) [@alexh](https://github.com/alexh) [@stuckinaboot](https://github.com/stuckinaboot) [@robzajac](https://github.com/robzajac)
[@zixuanzh](https://github.com/zixuanzh) [@alexh](https://github.com/alexh) [@stuckinaboot](https://github.com/stuckinaboot) [@robzajac](https://github.com/robzajac) [@carver](https://github.com/carver)
## Development
py-libp2p requires Python 3.7 and the best way to guarantee a clean Python 3.7 environment is with [`virtualenv`](https://virtualenv.pypa.io/en/stable/)
```sh
git clone git@github.com:libp2p/py-libp2p.git
cd py-libp2p
virtualenv -p python3.7 venv
. venv/bin/activate
pip3 install -r requirements_dev.txt
python setup.py develop
pip install -e .[dev]
```
## Testing
### Testing Setup
During development, you might like to have tests run on every file save.
Show flake8 errors on file change:
After installing our requirements (see above), you can:
```sh
cd tests
pytest
# Test flake8
when-changed -v -s -r -1 libp2p/ tests/ -c "clear; flake8 libp2p tests && echo 'flake8 success' || echo 'error'"
```
Run multi-process tests in one command, but without color:
```sh
# in the project root:
pytest --numprocesses=4 --looponfail --maxfail=1
# the same thing, succinctly:
pytest -n 4 -f --maxfail=1
```
Run in one thread, with color and desktop notifications:
```sh
cd venv
ptw --onfail "notify-send -t 5000 'Test failure ⚠⚠⚠⚠⚠' 'python 3 test on py-libp2p failed'" ../tests ../libp2p
```
Note that tests/libp2p/test_libp2p.py contains an end-to-end messaging test between two libp2p hosts, which is the bulk of our proof of concept.
### Release setup
Releases follow the same basic pattern as releases of some tangentially-related projects,
like Trinity. See [Trinity's release instructions](
https://trinity-client.readthedocs.io/en/latest/contributing.html#releasing).
## Requirements
The protobuf description in this repository was generated by `protoc` at version `3.7.1`.
## Feature Breakdown
py-libp2p aims for conformity with [the standard libp2p modules](https://github.com/libp2p/libp2p/blob/master/REQUIREMENTS.md#libp2p-modules-implementations). Below is a breakdown of the modules we have developed, are developing, and may develop in the future.
@ -46,16 +93,16 @@ py-libp2p aims for conformity with [the standard libp2p modules](https://github.
| Identify Protocol | Status |
| -------------------------------------------- | :-----------: |
| **`Identify`** | :tomato: |
| **`Identify`** | :lemon: |
| Transport Protocols | Status |
| -------------------------------------------- | :-----------: |
| **`TCP`** | :lemon: tests |
| **`TCP`** | :green_apple: |
| **`UDP`** | :tomato: |
| **`WebSockets`** | :tomato: |
| **`UTP`** | :tomato: |
| **`WebRTC`** | :tomato: |
| **`WebSockets`** | :chestnut: |
| **`UTP`** | :chestnut: |
| **`WebRTC`** | :chestnut: |
| **`SCTP`** | :chestnut: |
| **`Tor`** | :chestnut: |
| **`i2p`** | :chestnut: |
@ -68,7 +115,7 @@ py-libp2p aims for conformity with [the standard libp2p modules](https://github.
| Stream Muxers | Status |
| -------------------------------------------- | :-----------: |
| **`multiplex`** | :lemon: tests |
| **`multiplex`** | :green_apple: |
| **`yamux`** | :tomato: |
| **`benchmarks`** | :chestnut: |
| **`muxado`** | :chestnut: |
@ -85,39 +132,39 @@ py-libp2p aims for conformity with [the standard libp2p modules](https://github.
| Switch (Swarm) | Status |
| -------------------------------------------- | :-----------: |
| **`Switch`** | :lemon: tests |
| **`Dialer stack`** | :chestnut: |
| **`Switch`** | :green_apple: |
| **`Dialer stack`** | :green_apple: |
| Peer Discovery | Status |
| -------------------------------------------- | :-----------: |
| **`bootstrap list`** | :green_apple: |
| **`Kademlia DHT`** | :tomato: |
| **`mDNS`** | :tomato: |
| **`bootstrap list`** | :tomato: |
| **`Kademlia DHT`** | :chestnut: |
| **`mDNS`** | :chestnut: |
| **`PEX`** | :chestnut: |
| **`DNS`** | :chestnut: |
| Content Routing | Status |
| -------------------------------------------- | :-----------: |
| **`Kademlia DHT`** | :tomato: |
| **`floodsub`** | :tomato: |
| **`gossipsub`** | :tomato: |
| **`Kademlia DHT`** | :chestnut: |
| **`floodsub`** | :green_apple: |
| **`gossipsub`** | :green_apple: |
| **`PHT`** | :chestnut: |
| Peer Routing | Status |
| -------------------------------------------- | :-----------: |
| **`Kademlia DHT`** | :tomato: |
| **`floodsub`** | :tomato: |
| **`gossipsub`** | :tomato: |
| **`Kademlia DHT`** | :chestnut: |
| **`floodsub`** | :green_apple: |
| **`gossipsub`** | :green_apple: |
| **`PHT`** | :chestnut: |
| NAT Traversal | Status |
| -------------------------------------------- | :-----------: |
| **`nat-pmp`** | :tomato: |
| **`upnp`** | :tomato: |
| **`nat-pmp`** | :chestnut: |
| **`upnp`** | :chestnut: |
| **`ext addr discovery`** | :chestnut: |
| **`STUN-like`** | :chestnut: |
| **`line-switch relay`** | :chestnut: |

177
docs/Makefile Normal file
View File

@ -0,0 +1,177 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS = -W
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/web3.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/web3.qhc"
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/web3"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/web3"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

304
docs/conf.py Normal file
View File

@ -0,0 +1,304 @@
# -*- coding: utf-8 -*-
#
# py-libp2p documentation build configuration file, created by
# sphinx-quickstart on Thu Oct 16 20:43:24 2014.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
import os
DIR = os.path.dirname('__file__')
with open (os.path.join(DIR, '../setup.py'), 'r') as f:
for line in f:
if 'version=' in line:
setup_version = line.split('"')[1]
break
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.doctest',
'sphinx.ext.intersphinx',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = 'py-libp2p'
copyright = '2019, The Ethereum Foundation'
__version__ = setup_version
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '.'.join(__version__.split('.')[:2])
# The full version, including alpha/beta/rc tags.
release = __version__
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = [
'_build',
'modules.rst',
]
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'sphinx_rtd_theme'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'libp2pdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
('index', 'libp2p.tex', 'py-libp2p Documentation',
'The Ethereum Foundation', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'libp2p', 'py-libp2p Documentation',
['The Ethereum Foundation'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'py-libp2p', 'py-libp2p Documentation',
'The Ethereum Foundation', 'py-libp2p', 'The Python implementation of the libp2p networking stack',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
# -- Intersphinx configuration ------------------------------------------------
intersphinx_mapping = {
'python': ('https://docs.python.org/3.6', None),
}
# -- Doctest configuration ----------------------------------------
import doctest
doctest_default_flags = (0
| doctest.DONT_ACCEPT_TRUE_FOR_1
| doctest.ELLIPSIS
| doctest.IGNORE_EXCEPTION_DETAIL
| doctest.NORMALIZE_WHITESPACE
)
# -- Mocked dependencies ----------------------------------------
# Mock out dependencies that are unbuildable on readthedocs, as recommended here:
# https://docs.readthedocs.io/en/rel/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
import sys
from unittest.mock import MagicMock
# Add new modules to mock here (it should be the same list as those excluded in setup.py)
MOCK_MODULES = [
"fastecdsa",
"fastecdsa.encoding",
"fastecdsa.encoding.sec1",
]
sys.modules.update((mod_name, MagicMock()) for mod_name in MOCK_MODULES)

22
docs/examples.chat.rst Normal file
View File

@ -0,0 +1,22 @@
examples.chat package
=====================
Submodules
----------
examples.chat.chat module
-------------------------
.. automodule:: examples.chat.chat
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: examples.chat
:members:
:undoc-members:
:show-inheritance:

17
docs/examples.rst Normal file
View File

@ -0,0 +1,17 @@
examples package
================
Subpackages
-----------
.. toctree::
examples.chat
Module contents
---------------
.. automodule:: examples
:members:
:undoc-members:
:show-inheritance:

21
docs/index.rst Normal file
View File

@ -0,0 +1,21 @@
py-libp2p
==============================
The Python implementation of the libp2p networking stack
Contents
--------
.. toctree::
:maxdepth: 3
libp2p
release_notes
examples
Indices and tables
------------------
* :ref:`genindex`
* :ref:`modindex`

22
docs/libp2p.crypto.pb.rst Normal file
View File

@ -0,0 +1,22 @@
libp2p.crypto.pb package
========================
Submodules
----------
libp2p.crypto.pb.crypto\_pb2 module
-----------------------------------
.. automodule:: libp2p.crypto.pb.crypto_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.crypto.pb
:members:
:undoc-members:
:show-inheritance:

93
docs/libp2p.crypto.rst Normal file
View File

@ -0,0 +1,93 @@
libp2p.crypto package
=====================
Subpackages
-----------
.. toctree::
libp2p.crypto.pb
Submodules
----------
libp2p.crypto.authenticated\_encryption module
----------------------------------------------
.. automodule:: libp2p.crypto.authenticated_encryption
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.ecc module
------------------------
.. automodule:: libp2p.crypto.ecc
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.ed25519 module
----------------------------
.. automodule:: libp2p.crypto.ed25519
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.exceptions module
-------------------------------
.. automodule:: libp2p.crypto.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.key\_exchange module
----------------------------------
.. automodule:: libp2p.crypto.key_exchange
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.keys module
-------------------------
.. automodule:: libp2p.crypto.keys
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.rsa module
------------------------
.. automodule:: libp2p.crypto.rsa
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.secp256k1 module
------------------------------
.. automodule:: libp2p.crypto.secp256k1
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.serialization module
----------------------------------
.. automodule:: libp2p.crypto.serialization
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.crypto
:members:
:undoc-members:
:show-inheritance:

62
docs/libp2p.host.rst Normal file
View File

@ -0,0 +1,62 @@
libp2p.host package
===================
Submodules
----------
libp2p.host.basic\_host module
------------------------------
.. automodule:: libp2p.host.basic_host
:members:
:undoc-members:
:show-inheritance:
libp2p.host.defaults module
---------------------------
.. automodule:: libp2p.host.defaults
:members:
:undoc-members:
:show-inheritance:
libp2p.host.exceptions module
-----------------------------
.. automodule:: libp2p.host.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.host.host\_interface module
----------------------------------
.. automodule:: libp2p.host.host_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.host.ping module
-----------------------
.. automodule:: libp2p.host.ping
:members:
:undoc-members:
:show-inheritance:
libp2p.host.routed\_host module
-------------------------------
.. automodule:: libp2p.host.routed_host
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.host
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.identity.identify.pb package
===================================
Submodules
----------
libp2p.identity.identify.pb.identify\_pb2 module
------------------------------------------------
.. automodule:: libp2p.identity.identify.pb.identify_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.identity.identify.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,29 @@
libp2p.identity.identify package
================================
Subpackages
-----------
.. toctree::
libp2p.identity.identify.pb
Submodules
----------
libp2p.identity.identify.protocol module
----------------------------------------
.. automodule:: libp2p.identity.identify.protocol
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.identity.identify
:members:
:undoc-members:
:show-inheritance:

17
docs/libp2p.identity.rst Normal file
View File

@ -0,0 +1,17 @@
libp2p.identity package
=======================
Subpackages
-----------
.. toctree::
libp2p.identity.identify
Module contents
---------------
.. automodule:: libp2p.identity
:members:
:undoc-members:
:show-inheritance:

46
docs/libp2p.io.rst Normal file
View File

@ -0,0 +1,46 @@
libp2p.io package
=================
Submodules
----------
libp2p.io.abc module
--------------------
.. automodule:: libp2p.io.abc
:members:
:undoc-members:
:show-inheritance:
libp2p.io.exceptions module
---------------------------
.. automodule:: libp2p.io.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.io.msgio module
----------------------
.. automodule:: libp2p.io.msgio
:members:
:undoc-members:
:show-inheritance:
libp2p.io.utils module
----------------------
.. automodule:: libp2p.io.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.io
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,54 @@
libp2p.network.connection package
=================================
Submodules
----------
libp2p.network.connection.exceptions module
-------------------------------------------
.. automodule:: libp2p.network.connection.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.net\_connection\_interface module
-----------------------------------------------------------
.. automodule:: libp2p.network.connection.net_connection_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.raw\_connection module
------------------------------------------------
.. automodule:: libp2p.network.connection.raw_connection
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.raw\_connection\_interface module
-----------------------------------------------------------
.. automodule:: libp2p.network.connection.raw_connection_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.swarm\_connection module
--------------------------------------------------
.. automodule:: libp2p.network.connection.swarm_connection
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.network.connection
:members:
:undoc-members:
:show-inheritance:

54
docs/libp2p.network.rst Normal file
View File

@ -0,0 +1,54 @@
libp2p.network package
======================
Subpackages
-----------
.. toctree::
libp2p.network.connection
libp2p.network.stream
Submodules
----------
libp2p.network.exceptions module
--------------------------------
.. automodule:: libp2p.network.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.network.network\_interface module
----------------------------------------
.. automodule:: libp2p.network.network_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.notifee\_interface module
----------------------------------------
.. automodule:: libp2p.network.notifee_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.swarm module
---------------------------
.. automodule:: libp2p.network.swarm
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.network
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,38 @@
libp2p.network.stream package
=============================
Submodules
----------
libp2p.network.stream.exceptions module
---------------------------------------
.. automodule:: libp2p.network.stream.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.network.stream.net\_stream module
----------------------------------------
.. automodule:: libp2p.network.stream.net_stream
:members:
:undoc-members:
:show-inheritance:
libp2p.network.stream.net\_stream\_interface module
---------------------------------------------------
.. automodule:: libp2p.network.stream.net_stream_interface
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.network.stream
:members:
:undoc-members:
:show-inheritance:

78
docs/libp2p.peer.rst Normal file
View File

@ -0,0 +1,78 @@
libp2p.peer package
===================
Submodules
----------
libp2p.peer.addrbook\_interface module
--------------------------------------
.. automodule:: libp2p.peer.addrbook_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.id module
---------------------
.. automodule:: libp2p.peer.id
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerdata module
---------------------------
.. automodule:: libp2p.peer.peerdata
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerdata\_interface module
--------------------------------------
.. automodule:: libp2p.peer.peerdata_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerinfo module
---------------------------
.. automodule:: libp2p.peer.peerinfo
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peermetadata\_interface module
------------------------------------------
.. automodule:: libp2p.peer.peermetadata_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerstore module
----------------------------
.. automodule:: libp2p.peer.peerstore
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerstore\_interface module
---------------------------------------
.. automodule:: libp2p.peer.peerstore_interface
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.peer
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,70 @@
libp2p.protocol\_muxer package
==============================
Submodules
----------
libp2p.protocol\_muxer.exceptions module
----------------------------------------
.. automodule:: libp2p.protocol_muxer.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect module
-----------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_client module
-------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_client
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_client\_interface module
------------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_client_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_communicator module
-------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_communicator
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_communicator\_interface module
------------------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_communicator_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_muxer\_interface module
-----------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_muxer_interface
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.protocol_muxer
:members:
:undoc-members:
:show-inheritance:

22
docs/libp2p.pubsub.pb.rst Normal file
View File

@ -0,0 +1,22 @@
libp2p.pubsub.pb package
========================
Submodules
----------
libp2p.pubsub.pb.rpc\_pb2 module
--------------------------------
.. automodule:: libp2p.pubsub.pb.rpc_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.pubsub.pb
:members:
:undoc-members:
:show-inheritance:

93
docs/libp2p.pubsub.rst Normal file
View File

@ -0,0 +1,93 @@
libp2p.pubsub package
=====================
Subpackages
-----------
.. toctree::
libp2p.pubsub.pb
Submodules
----------
libp2p.pubsub.abc module
------------------------
.. automodule:: libp2p.pubsub.abc
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.exceptions module
-------------------------------
.. automodule:: libp2p.pubsub.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.floodsub module
-----------------------------
.. automodule:: libp2p.pubsub.floodsub
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.gossipsub module
------------------------------
.. automodule:: libp2p.pubsub.gossipsub
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.mcache module
---------------------------
.. automodule:: libp2p.pubsub.mcache
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.pubsub module
---------------------------
.. automodule:: libp2p.pubsub.pubsub
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.pubsub\_notifee module
------------------------------------
.. automodule:: libp2p.pubsub.pubsub_notifee
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.subscription module
---------------------------------
.. automodule:: libp2p.pubsub.subscription
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.validators module
-------------------------------
.. automodule:: libp2p.pubsub.validators
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.pubsub
:members:
:undoc-members:
:show-inheritance:

23
docs/libp2p.routing.rst Normal file
View File

@ -0,0 +1,23 @@
libp2p.routing package
======================
Submodules
----------
libp2p.routing.interfaces module
--------------------------------
.. automodule:: libp2p.routing.interfaces
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.routing
:members:
:undoc-members:
:show-inheritance:

57
docs/libp2p.rst Normal file
View File

@ -0,0 +1,57 @@
libp2p package
==============
Subpackages
-----------
.. toctree::
libp2p.crypto
libp2p.host
libp2p.identity
libp2p.io
libp2p.network
libp2p.peer
libp2p.protocol_muxer
libp2p.pubsub
libp2p.routing
libp2p.security
libp2p.stream_muxer
libp2p.tools
libp2p.transport
Submodules
----------
libp2p.exceptions module
------------------------
.. automodule:: libp2p.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.typing module
--------------------
.. automodule:: libp2p.typing
:members:
:undoc-members:
:show-inheritance:
libp2p.utils module
-------------------
.. automodule:: libp2p.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.security.insecure.pb package
===================================
Submodules
----------
libp2p.security.insecure.pb.plaintext\_pb2 module
-------------------------------------------------
.. automodule:: libp2p.security.insecure.pb.plaintext_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.insecure.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,29 @@
libp2p.security.insecure package
================================
Subpackages
-----------
.. toctree::
libp2p.security.insecure.pb
Submodules
----------
libp2p.security.insecure.transport module
-----------------------------------------
.. automodule:: libp2p.security.insecure.transport
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.insecure
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.security.noise.pb package
================================
Submodules
----------
libp2p.security.noise.pb.noise\_pb2 module
------------------------------------------
.. automodule:: libp2p.security.noise.pb.noise_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.noise.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,61 @@
libp2p.security.noise package
=============================
Subpackages
-----------
.. toctree::
libp2p.security.noise.pb
Submodules
----------
libp2p.security.noise.exceptions module
---------------------------------------
.. automodule:: libp2p.security.noise.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.io module
-------------------------------
.. automodule:: libp2p.security.noise.io
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.messages module
-------------------------------------
.. automodule:: libp2p.security.noise.messages
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.patterns module
-------------------------------------
.. automodule:: libp2p.security.noise.patterns
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.transport module
--------------------------------------
.. automodule:: libp2p.security.noise.transport
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.noise
:members:
:undoc-members:
:show-inheritance:

71
docs/libp2p.security.rst Normal file
View File

@ -0,0 +1,71 @@
libp2p.security package
=======================
Subpackages
-----------
.. toctree::
libp2p.security.insecure
libp2p.security.noise
libp2p.security.secio
Submodules
----------
libp2p.security.base\_session module
------------------------------------
.. automodule:: libp2p.security.base_session
:members:
:undoc-members:
:show-inheritance:
libp2p.security.base\_transport module
--------------------------------------
.. automodule:: libp2p.security.base_transport
:members:
:undoc-members:
:show-inheritance:
libp2p.security.exceptions module
---------------------------------
.. automodule:: libp2p.security.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.security.secure\_conn\_interface module
----------------------------------------------
.. automodule:: libp2p.security.secure_conn_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.security.secure\_transport\_interface module
---------------------------------------------------
.. automodule:: libp2p.security.secure_transport_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.security.security\_multistream module
--------------------------------------------
.. automodule:: libp2p.security.security_multistream
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.security.secio.pb package
================================
Submodules
----------
libp2p.security.secio.pb.spipe\_pb2 module
------------------------------------------
.. automodule:: libp2p.security.secio.pb.spipe_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.secio.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,37 @@
libp2p.security.secio package
=============================
Subpackages
-----------
.. toctree::
libp2p.security.secio.pb
Submodules
----------
libp2p.security.secio.exceptions module
---------------------------------------
.. automodule:: libp2p.security.secio.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.security.secio.transport module
--------------------------------------
.. automodule:: libp2p.security.secio.transport
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.secio
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,54 @@
libp2p.stream\_muxer.mplex package
==================================
Submodules
----------
libp2p.stream\_muxer.mplex.constants module
-------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.constants
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.datastructures module
------------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.datastructures
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.exceptions module
--------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.mplex module
---------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.mplex
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.mplex\_stream module
-----------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.mplex_stream
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.stream_muxer.mplex
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,45 @@
libp2p.stream\_muxer package
============================
Subpackages
-----------
.. toctree::
libp2p.stream_muxer.mplex
Submodules
----------
libp2p.stream\_muxer.abc module
-------------------------------
.. automodule:: libp2p.stream_muxer.abc
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.exceptions module
--------------------------------------
.. automodule:: libp2p.stream_muxer.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.muxer\_multistream module
----------------------------------------------
.. automodule:: libp2p.stream_muxer.muxer_multistream
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.stream_muxer
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,38 @@
libp2p.tools.pubsub package
===========================
Submodules
----------
libp2p.tools.pubsub.dummy\_account\_node module
-----------------------------------------------
.. automodule:: libp2p.tools.pubsub.dummy_account_node
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.pubsub.floodsub\_integration\_test\_settings module
----------------------------------------------------------------
.. automodule:: libp2p.tools.pubsub.floodsub_integration_test_settings
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.pubsub.utils module
--------------------------------
.. automodule:: libp2p.tools.pubsub.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.tools.pubsub
:members:
:undoc-members:
:show-inheritance:

47
docs/libp2p.tools.rst Normal file
View File

@ -0,0 +1,47 @@
libp2p.tools package
====================
Subpackages
-----------
.. toctree::
libp2p.tools.pubsub
The interop module is left out for now, because of the extra dependencies it requires.
Submodules
----------
libp2p.tools.constants module
-----------------------------
.. automodule:: libp2p.tools.constants
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.factories module
-----------------------------
.. automodule:: libp2p.tools.factories
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.utils module
-------------------------
.. automodule:: libp2p.tools.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.tools
:members:
:undoc-members:
:show-inheritance:

61
docs/libp2p.transport.rst Normal file
View File

@ -0,0 +1,61 @@
libp2p.transport package
========================
Subpackages
-----------
.. toctree::
libp2p.transport.tcp
Submodules
----------
libp2p.transport.exceptions module
----------------------------------
.. automodule:: libp2p.transport.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.listener\_interface module
-------------------------------------------
.. automodule:: libp2p.transport.listener_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.transport\_interface module
--------------------------------------------
.. automodule:: libp2p.transport.transport_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.typing module
------------------------------
.. automodule:: libp2p.transport.typing
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.upgrader module
--------------------------------
.. automodule:: libp2p.transport.upgrader
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.transport
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.transport.tcp package
============================
Submodules
----------
libp2p.transport.tcp.tcp module
-------------------------------
.. automodule:: libp2p.transport.tcp.tcp
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.transport.tcp
:members:
:undoc-members:
:show-inheritance:

95
docs/release_notes.rst Normal file
View File

@ -0,0 +1,95 @@
Release Notes
=============
.. towncrier release notes start
libp2p v0.1.5 (2020-03-25)
---------------------------
Features
~~~~~~~~
- Dial all multiaddrs stored for a peer when attempting to connect (not just the first one in the peer store). (`#386 <https://github.com/libp2p/py-libp2p/issues/386>`__)
- Migrate transport stack to trio-compatible code. Merge in #404. (`#396 <https://github.com/libp2p/py-libp2p/issues/396>`__)
- Migrate network stack to trio-compatible code. Merge in #404. (`#397 <https://github.com/libp2p/py-libp2p/issues/397>`__)
- Migrate host, peer and protocols stacks to trio-compatible code. Merge in #404. (`#398 <https://github.com/libp2p/py-libp2p/issues/398>`__)
- Migrate muxer and security transport stacks to trio-compatible code. Merge in #404. (`#399 <https://github.com/libp2p/py-libp2p/issues/399>`__)
- Migrate pubsub stack to trio-compatible code. Merge in #404. (`#400 <https://github.com/libp2p/py-libp2p/issues/400>`__)
- Fix interop tests w/ new trio-style code. Merge in #404. (`#401 <https://github.com/libp2p/py-libp2p/issues/401>`__)
- Fix remainder of test code w/ new trio-style code. Merge in #404. (`#402 <https://github.com/libp2p/py-libp2p/issues/402>`__)
- Add initial infrastructure for `noise` security transport. (`#405 <https://github.com/libp2p/py-libp2p/issues/405>`__)
- Add `PatternXX` of `noise` security transport. (`#406 <https://github.com/libp2p/py-libp2p/issues/406>`__)
- The `msg_id` in a pubsub message is now configurable by the user of the library. (`#410 <https://github.com/libp2p/py-libp2p/issues/410>`__)
Bugfixes
~~~~~~~~
- Use `sha256` when calculating a peer's ID from their public key in Kademlia DHTs. (`#385 <https://github.com/libp2p/py-libp2p/issues/385>`__)
- Store peer ids in ``set`` instead of ``list`` and check if peer id exists in ``dict`` before accessing to prevent ``KeyError``. (`#387 <https://github.com/libp2p/py-libp2p/issues/387>`__)
- Do not close a connection if it has been reset. (`#394 <https://github.com/libp2p/py-libp2p/issues/394>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Add support for `fastecdsa` on windows (and thereby supporting windows installation via `pip`) (`#380 <https://github.com/libp2p/py-libp2p/issues/380>`__)
- Prefer f-string style formatting everywhere except logging statements. (`#389 <https://github.com/libp2p/py-libp2p/issues/389>`__)
- Mark `lru` dependency as third-party to fix a windows inconsistency. (`#392 <https://github.com/libp2p/py-libp2p/issues/392>`__)
- Bump `multiaddr` dependency to version `0.0.9` so that multiaddr objects are hashable. (`#393 <https://github.com/libp2p/py-libp2p/issues/393>`__)
- Remove incremental mode of mypy to disable some warnings. (`#403 <https://github.com/libp2p/py-libp2p/issues/403>`__)
libp2p v0.1.4 (2019-12-12)
--------------------------
Features
~~~~~~~~
- Added support for Python 3.6 (`#372 <https://github.com/libp2p/py-libp2p/issues/372>`__)
- Add signing and verification to pubsub (`#362 <https://github.com/libp2p/py-libp2p/issues/362>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Refactor and cleanup gossipsub (`#373 <https://github.com/libp2p/py-libp2p/issues/373>`__)
libp2p v0.1.3 (2019-11-27)
--------------------------
Bugfixes
~~~~~~~~
- Handle Stream* errors (like ``StreamClosed``) during calls to ``stream.write()`` and
``stream.read()`` (`#350 <https://github.com/libp2p/py-libp2p/issues/350>`__)
- Relax the protobuf dependency to play nicely with other libraries. It was pinned to 3.9.0, and now
permits v3.10 up to (but not including) v4. (`#354 <https://github.com/libp2p/py-libp2p/issues/354>`__)
- Fixes KeyError when peer in a stream accidentally closes and resets the stream, because handlers
for both will try to ``del streams[stream_id]`` without checking if the entry still exists. (`#355 <https://github.com/libp2p/py-libp2p/issues/355>`__)
Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~
- Use Sphinx & autodoc to generate docs, now available on `py-libp2p.readthedocs.io <https://py-libp2p.readthedocs.io>`_ (`#318 <https://github.com/libp2p/py-libp2p/issues/318>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Added Makefile target to test a packaged version of libp2p before release. (`#353 <https://github.com/libp2p/py-libp2p/issues/353>`__)
- Move helper tools from ``tests/`` to ``libp2p/tools/``, and some mildly-related cleanups. (`#356 <https://github.com/libp2p/py-libp2p/issues/356>`__)
Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~
- `#357 <https://github.com/libp2p/py-libp2p/issues/357>`__
v0.1.2
--------------
Welcome to the great beyond, where changes were not tracked by release...

View File

@ -1,92 +1,103 @@
import asyncio
import argparse
import sys
import click
import multiaddr
import trio
from libp2p import new_node
from libp2p import new_host
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.typing import TProtocol
PROTOCOL_ID = TProtocol("/chat/1.0.0")
MAX_READ_LEN = 2 ** 32 - 1
PROTOCOL_ID = '/chat/1.0.0'
async def read_data(stream):
async def read_data(stream: INetStream) -> None:
while True:
read_string = await stream.read()
if read_string is not None:
read_string = read_string.decode()
read_bytes = await stream.read(MAX_READ_LEN)
if read_bytes is not None:
read_string = read_bytes.decode()
if read_string != "\n":
# Green console colour: \x1b[32m
# Reset console colour: \x1b[0m
print("\x1b[32m %s\x1b[0m " % read_string, end="")
async def write_data(stream):
loop = asyncio.get_event_loop()
async def write_data(stream: INetStream) -> None:
async_f = trio.wrap_file(sys.stdin)
while True:
line = await loop.run_in_executor(None, sys.stdin.readline)
line = await async_f.readline()
await stream.write(line.encode())
async def run(port, destination):
host = await new_node(transport_opt=["/ip4/127.0.0.1/tcp/%s" % port])
if not destination: # its the server
async def stream_handler(stream):
asyncio.ensure_future(read_data(stream))
asyncio.ensure_future(write_data(stream))
host.set_stream_handler(PROTOCOL_ID, stream_handler)
async def run(port: int, destination: str) -> None:
localhost_ip = "127.0.0.1"
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
host = new_host()
async with host.run(listen_addrs=[listen_addr]), trio.open_nursery() as nursery:
if not destination: # its the server
port = None
for listener in host.network.listeners.values():
for addr in listener.get_addrs():
port = int(addr.value_for_protocol('tcp'))
async def stream_handler(stream: INetStream) -> None:
nursery.start_soon(read_data, stream)
nursery.start_soon(write_data, stream)
if not port:
raise RuntimeError("was not able to find the actual local port")
host.set_stream_handler(PROTOCOL_ID, stream_handler)
print("Run './examples/chat/chat.py -p %s -d /ip4/127.0.0.1/tcp/%s/p2p/%s' on another console.\n" %
(int(port) + 1, port, host.get_id().pretty()))
print("You can replace 127.0.0.1 with public IP as well.")
print("\nWaiting for incoming connection\n\n")
print(
f"Run 'python ./examples/chat/chat.py "
f"-p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id().pretty()}' "
"on another console."
)
print("Waiting for incoming connection...")
else: # its the client
m = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(m)
# Associate the peer with local ip address
await host.connect(info)
else: # its the client
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
# Associate the peer with local ip address
await host.connect(info)
# Start a stream with the destination.
# Multiaddress of the destination peer is fetched from the peerstore using 'peerId'.
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
# Start a stream with the destination.
# Multiaddress of the destination peer is fetched from the peerstore using 'peerId'.
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
nursery.start_soon(read_data, stream)
nursery.start_soon(write_data, stream)
print(f"Connected to peer {info.addrs[0]}")
asyncio.ensure_future(read_data(stream))
asyncio.ensure_future(write_data(stream))
print("Connected to peer %s" % info.addrs[0])
await trio.sleep_forever()
@click.command()
@click.option('--port', '-p', help='source port number', default=8000)
@click.option('--destination', '-d', help="Destination multiaddr string")
@click.option('--help', is_flag=True, default=False, help='display help')
# @click.option('--debug', is_flag=True, default=False, help='Debug generates the same node ID on every execution')
def main(port, destination, help):
def main() -> None:
description = """
This program demonstrates a simple p2p chat application using libp2p.
To use it, first run 'python ./chat -p <PORT>', where <PORT> is the port number.
Then, run another host with 'python ./chat -p <ANOTHER_PORT> -d <DESTINATION>',
where <DESTINATION> is the multiaddress of the previous listener host.
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p", "--port", default=8000, type=int, help="source port number"
)
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
args = parser.parse_args()
if help:
print("This program demonstrates a simple p2p chat application using libp2p\n\n")
print("Usage: Run './chat -p <SOURCE_PORT>' where <SOURCE_PORT> can be any port number.")
print("Now run './chat -p <PORT> -d <MULTIADDR>' where <MULTIADDR> is multiaddress of previous listener host.")
return
if not args.port:
raise RuntimeError("was not able to determine a local port")
loop = asyncio.get_event_loop()
try:
asyncio.ensure_future(run(port, destination))
loop.run_forever()
trio.run(run, *(args.port, args.destination))
except KeyboardInterrupt:
pass
finally:
loop.close()
if __name__ == '__main__':
if __name__ == "__main__":
main()

116
examples/echo/echo.py Normal file
View File

@ -0,0 +1,116 @@
import argparse
import multiaddr
import trio
from libp2p import new_host
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.typing import TProtocol
PROTOCOL_ID = TProtocol("/echo/1.0.0")
async def _echo_stream_handler(stream: INetStream) -> None:
# Wait until EOF
msg = await stream.read()
await stream.write(msg)
await stream.close()
async def run(port: int, destination: str, seed: int = None) -> None:
localhost_ip = "127.0.0.1"
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
if seed:
import random
random.seed(seed)
secret_number = random.getrandbits(32 * 8)
secret = secret_number.to_bytes(length=32, byteorder="big")
else:
import secrets
secret = secrets.token_bytes(32)
host = new_host(key_pair=create_new_key_pair(secret))
async with host.run(listen_addrs=[listen_addr]):
print(f"I am {host.get_id().to_string()}")
if not destination: # its the server
host.set_stream_handler(PROTOCOL_ID, _echo_stream_handler)
print(
f"Run 'python ./examples/echo/echo.py "
f"-p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id().pretty()}' "
"on another console."
)
print("Waiting for incoming connections...")
await trio.sleep_forever()
else: # its the client
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
# Associate the peer with local ip address
await host.connect(info)
# Start a stream with the destination.
# Multiaddress of the destination peer is fetched from the peerstore using 'peerId'.
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
msg = b"hi, there!\n"
await stream.write(msg)
# Notify the other side about EOF
await stream.close()
response = await stream.read()
print(f"Sent: {msg}")
print(f"Got: {response}")
def main() -> None:
description = """
This program demonstrates a simple echo protocol where a peer listens for
connections and copies back any input received on a stream.
To use it, first run 'python ./echo -p <PORT>', where <PORT> is the port number.
Then, run another host with 'python ./chat -p <ANOTHER_PORT> -d <DESTINATION>',
where <DESTINATION> is the multiaddress of the previous listener host.
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p", "--port", default=8000, type=int, help="source port number"
)
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
parser.add_argument(
"-s",
"--seed",
type=int,
help="provide a seed to the random number generator (e.g. to fix peer IDs across runs)",
)
args = parser.parse_args()
if not args.port:
raise RuntimeError("was not able to determine a local port")
try:
trio.run(run, args.port, args.destination, args.seed)
except KeyboardInterrupt:
pass
if __name__ == "__main__":
main()

View File

@ -1,38 +1,98 @@
from Crypto.PublicKey import RSA
import multiaddr
from .peer.peerstore import PeerStore
from .peer.id import id_from_public_key
from .network.swarm import Swarm
from .host.basic_host import BasicHost
from .transport.upgrader import TransportUpgrader
from .transport.tcp.tcp import TCP
from libp2p.crypto.keys import KeyPair
from libp2p.crypto.rsa import create_new_key_pair
from libp2p.host.basic_host import BasicHost
from libp2p.host.host_interface import IHost
from libp2p.host.routed_host import RoutedHost
from libp2p.network.network_interface import INetworkService
from libp2p.network.swarm import Swarm
from libp2p.peer.id import ID
from libp2p.peer.peerstore import PeerStore
from libp2p.peer.peerstore_interface import IPeerStore
from libp2p.routing.interfaces import IPeerRouting
from libp2p.security.insecure.transport import PLAINTEXT_PROTOCOL_ID, InsecureTransport
import libp2p.security.secio.transport as secio
from libp2p.stream_muxer.mplex.mplex import MPLEX_PROTOCOL_ID, Mplex
from libp2p.transport.tcp.tcp import TCP
from libp2p.transport.typing import TMuxerOptions, TSecurityOptions
from libp2p.transport.upgrader import TransportUpgrader
from libp2p.typing import TProtocol
async def new_node(
id_opt=None, transport_opt=None,
muxer_opt=None, sec_opt=None, peerstore=None):
def generate_new_rsa_identity() -> KeyPair:
return create_new_key_pair()
if id_opt is None:
new_key = RSA.generate(2048, e=65537)
id_opt = id_from_public_key(new_key.publickey())
# private_key = new_key.exportKey("PEM")
transport_opt = transport_opt or ["/ip4/127.0.0.1/tcp/8001"]
transport_opt = [multiaddr.Multiaddr(t) for t in transport_opt]
muxer_opt = muxer_opt or ["mplex/6.7.0"]
sec_opt = sec_opt or ["secio"]
peerstore = peerstore or PeerStore()
def generate_peer_id_from(key_pair: KeyPair) -> ID:
public_key = key_pair.public_key
return ID.from_pubkey(public_key)
upgrader = TransportUpgrader(sec_opt, transport_opt)
swarm = Swarm(id_opt, peerstore, upgrader)
tcp = TCP()
swarm.add_transport(tcp)
await swarm.listen(transport_opt[0])
# TODO enable support for other host type
# TODO routing unimplemented
host = BasicHost(swarm)
def new_swarm(
key_pair: KeyPair = None,
muxer_opt: TMuxerOptions = None,
sec_opt: TSecurityOptions = None,
peerstore_opt: IPeerStore = None,
) -> INetworkService:
"""
Create a swarm instance based on the parameters.
:param key_pair: optional choice of the ``KeyPair``
:param muxer_opt: optional choice of stream muxer
:param sec_opt: optional choice of security upgrade
:param peerstore_opt: optional peerstore
:return: return a default swarm instance
"""
if key_pair is None:
key_pair = generate_new_rsa_identity()
id_opt = generate_peer_id_from(key_pair)
# TODO: Parse `listen_addrs` to determine transport
transport = TCP()
muxer_transports_by_protocol = muxer_opt or {MPLEX_PROTOCOL_ID: Mplex}
security_transports_by_protocol = sec_opt or {
TProtocol(PLAINTEXT_PROTOCOL_ID): InsecureTransport(key_pair),
TProtocol(secio.ID): secio.Transport(key_pair),
}
upgrader = TransportUpgrader(
security_transports_by_protocol, muxer_transports_by_protocol
)
peerstore = peerstore_opt or PeerStore()
# Store our key pair in peerstore
peerstore.add_key_pair(id_opt, key_pair)
return Swarm(id_opt, peerstore, upgrader, transport)
def new_host(
key_pair: KeyPair = None,
muxer_opt: TMuxerOptions = None,
sec_opt: TSecurityOptions = None,
peerstore_opt: IPeerStore = None,
disc_opt: IPeerRouting = None,
) -> IHost:
"""
Create a new libp2p host based on the given parameters.
:param key_pair: optional choice of the ``KeyPair``
:param muxer_opt: optional choice of stream muxer
:param sec_opt: optional choice of security upgrade
:param peerstore_opt: optional peerstore
:param disc_opt: optional discovery
:return: return a host instance
"""
swarm = new_swarm(
key_pair=key_pair,
muxer_opt=muxer_opt,
sec_opt=sec_opt,
peerstore_opt=peerstore_opt,
)
host: IHost
if disc_opt:
host = RoutedHost(swarm, disc_opt)
else:
host = BasicHost(swarm)
return host

View File

@ -0,0 +1,125 @@
from dataclasses import dataclass
import hmac
from typing import Tuple
from Crypto.Cipher import AES
import Crypto.Util.Counter as Counter
class InvalidMACException(Exception):
pass
@dataclass(frozen=True)
class EncryptionParameters:
cipher_type: str
hash_type: str
iv: bytes
mac_key: bytes
cipher_key: bytes
class MacAndCipher:
def __init__(self, parameters: EncryptionParameters) -> None:
self.authenticator = hmac.new(
parameters.mac_key, digestmod=parameters.hash_type
)
iv_bit_size = 8 * len(parameters.iv)
cipher = AES.new(
parameters.cipher_key,
AES.MODE_CTR,
counter=Counter.new(
iv_bit_size,
initial_value=int.from_bytes(parameters.iv, byteorder="big"),
),
)
self.cipher = cipher
def encrypt(self, data: bytes) -> bytes:
return self.cipher.encrypt(data)
def authenticate(self, data: bytes) -> bytes:
authenticator = self.authenticator.copy()
authenticator.update(data)
return authenticator.digest()
def decrypt_if_valid(self, data_with_tag: bytes) -> bytes:
tag_position = len(data_with_tag) - self.authenticator.digest_size
data = data_with_tag[:tag_position]
tag = data_with_tag[tag_position:]
authenticator = self.authenticator.copy()
authenticator.update(data)
expected_tag = authenticator.digest()
if not hmac.compare_digest(tag, expected_tag):
raise InvalidMACException(expected_tag, tag)
return self.cipher.decrypt(data)
def initialize_pair(
cipher_type: str, hash_type: str, secret: bytes
) -> Tuple[EncryptionParameters, EncryptionParameters]:
"""Return a pair of ``Keys`` for use in securing a communications channel
with authenticated encryption derived from the ``secret`` and using the
requested ``cipher_type`` and ``hash_type``."""
if cipher_type != "AES-128":
raise NotImplementedError()
if hash_type != "SHA256":
raise NotImplementedError()
iv_size = 16
cipher_key_size = 16
hmac_key_size = 20
seed = "key expansion".encode()
params_size = iv_size + cipher_key_size + hmac_key_size
result = bytearray(2 * params_size)
authenticator = hmac.new(secret, digestmod=hash_type)
authenticator.update(seed)
tag = authenticator.digest()
i = 0
len_result = 2 * params_size
while i < len_result:
authenticator = hmac.new(secret, digestmod=hash_type)
authenticator.update(tag)
authenticator.update(seed)
another_tag = authenticator.digest()
remaining_bytes = len(another_tag)
if i + remaining_bytes > len_result:
remaining_bytes = len_result - i
result[i : i + remaining_bytes] = another_tag[0:remaining_bytes]
i += remaining_bytes
authenticator = hmac.new(secret, digestmod=hash_type)
authenticator.update(tag)
tag = authenticator.digest()
first_half = result[:params_size]
second_half = result[params_size:]
return (
EncryptionParameters(
cipher_type,
hash_type,
first_half[0:iv_size],
first_half[iv_size + cipher_key_size :],
first_half[iv_size : iv_size + cipher_key_size],
),
EncryptionParameters(
cipher_type,
hash_type,
second_half[0:iv_size],
second_half[iv_size + cipher_key_size :],
second_half[iv_size : iv_size + cipher_key_size],
),
)

68
libp2p/crypto/ecc.py Normal file
View File

@ -0,0 +1,68 @@
from fastecdsa import curve as curve_types
from fastecdsa import keys, point
from fastecdsa.encoding.sec1 import SEC1Encoder
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
def infer_local_type(curve: str) -> curve_types.Curve:
"""converts a ``str`` representation of some elliptic curve to a
representation understood by the backend of this module."""
if curve == "P-256":
return curve_types.P256
else:
raise NotImplementedError()
class ECCPublicKey(PublicKey):
def __init__(self, impl: point.Point, curve: curve_types.Curve) -> None:
self.impl = impl
self.curve = curve
def to_bytes(self) -> bytes:
return SEC1Encoder.encode_public_key(self.impl, compressed=False)
@classmethod
def from_bytes(cls, data: bytes, curve: str) -> "ECCPublicKey":
curve_type = infer_local_type(curve)
public_key_impl = SEC1Encoder.decode_public_key(data, curve_type)
return cls(public_key_impl, curve_type)
def get_type(self) -> KeyType:
return KeyType.ECC_P256
def verify(self, data: bytes, signature: bytes) -> bool:
raise NotImplementedError()
class ECCPrivateKey(PrivateKey):
def __init__(self, impl: int, curve: curve_types.Curve) -> None:
self.impl = impl
self.curve = curve
@classmethod
def new(cls, curve: str) -> "ECCPrivateKey":
curve_type = infer_local_type(curve)
private_key_impl = keys.gen_private_key(curve_type)
return cls(private_key_impl, curve_type)
def to_bytes(self) -> bytes:
return keys.export_key(self.impl, self.curve)
def get_type(self) -> KeyType:
return KeyType.ECC_P256
def sign(self, data: bytes) -> bytes:
raise NotImplementedError()
def get_public_key(self) -> PublicKey:
public_key_impl = keys.get_public_key(self.impl, self.curve)
return ECCPublicKey(public_key_impl, self.curve)
def create_new_key_pair(curve: str) -> KeyPair:
"""Return a new ECC keypair with the requested ``curve`` type, e.g.
"P-256"."""
private_key = ECCPrivateKey.new(curve)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

69
libp2p/crypto/ed25519.py Normal file
View File

@ -0,0 +1,69 @@
from Crypto.Hash import SHA256
from nacl.exceptions import BadSignatureError
from nacl.public import PrivateKey as PrivateKeyImpl
from nacl.public import PublicKey as PublicKeyImpl
from nacl.signing import SigningKey, VerifyKey
import nacl.utils as utils
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
class Ed25519PublicKey(PublicKey):
def __init__(self, impl: PublicKeyImpl) -> None:
self.impl = impl
def to_bytes(self) -> bytes:
return bytes(self.impl)
@classmethod
def from_bytes(cls, key_bytes: bytes) -> "Ed25519PublicKey":
return cls(PublicKeyImpl(key_bytes))
def get_type(self) -> KeyType:
return KeyType.Ed25519
def verify(self, data: bytes, signature: bytes) -> bool:
verify_key = VerifyKey(self.to_bytes())
try:
verify_key.verify(data, signature)
except BadSignatureError:
return False
return True
class Ed25519PrivateKey(PrivateKey):
def __init__(self, impl: PrivateKeyImpl) -> None:
self.impl = impl
@classmethod
def new(cls, seed: bytes = None) -> "Ed25519PrivateKey":
if not seed:
seed = utils.random()
private_key_impl = PrivateKeyImpl.from_seed(seed)
return cls(private_key_impl)
def to_bytes(self) -> bytes:
return bytes(self.impl)
@classmethod
def from_bytes(cls, data: bytes) -> "Ed25519PrivateKey":
impl = PrivateKeyImpl(data)
return cls(impl)
def get_type(self) -> KeyType:
return KeyType.Ed25519
def sign(self, data: bytes) -> bytes:
h = SHA256.new(data)
signing_key = SigningKey(self.to_bytes())
return signing_key.sign(h)
def get_public_key(self) -> PublicKey:
return Ed25519PublicKey(self.impl.public_key)
def create_new_key_pair(seed: bytes = None) -> KeyPair:
private_key = Ed25519PrivateKey.new(seed)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

View File

@ -0,0 +1,12 @@
from libp2p.exceptions import BaseLibp2pError
class CryptographyError(BaseLibp2pError):
pass
class MissingDeserializerError(CryptographyError):
"""Raise if the requested deserialization routine is missing for some type
of cryptographic key."""
pass

View File

@ -0,0 +1,29 @@
from typing import Callable, Tuple, cast
from fastecdsa.encoding import util
from libp2p.crypto.ecc import ECCPrivateKey, ECCPublicKey, create_new_key_pair
from libp2p.crypto.keys import PublicKey
SharedKeyGenerator = Callable[[bytes], bytes]
int_bytelen = util.int_bytelen
def create_ephemeral_key_pair(curve_type: str) -> Tuple[PublicKey, SharedKeyGenerator]:
"""Facilitates ECDH key exchange."""
if curve_type != "P-256":
raise NotImplementedError()
key_pair = create_new_key_pair(curve_type)
def _key_exchange(serialized_remote_public_key: bytes) -> bytes:
private_key = cast(ECCPrivateKey, key_pair.private_key)
remote_point = ECCPublicKey.from_bytes(serialized_remote_public_key, curve_type)
secret_point = remote_point.impl * private_key.impl
secret_x_coordinate = secret_point.x
byte_size = int_bytelen(secret_x_coordinate)
return secret_x_coordinate.to_bytes(byte_size, byteorder="big")
return key_pair.public_key, _key_exchange

91
libp2p/crypto/keys.py Normal file
View File

@ -0,0 +1,91 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
from enum import Enum, unique
from .pb import crypto_pb2 as protobuf
@unique
class KeyType(Enum):
RSA = 0
Ed25519 = 1
Secp256k1 = 2
ECDSA = 3
ECC_P256 = 4
class Key(ABC):
"""A ``Key`` represents a cryptographic key."""
@abstractmethod
def to_bytes(self) -> bytes:
"""Returns the byte representation of this key."""
...
@abstractmethod
def get_type(self) -> KeyType:
"""Returns the ``KeyType`` for ``self``."""
...
def __eq__(self, other: object) -> bool:
if not isinstance(other, Key):
return NotImplemented
return self.to_bytes() == other.to_bytes()
class PublicKey(Key):
"""A ``PublicKey`` represents a cryptographic public key."""
@abstractmethod
def verify(self, data: bytes, signature: bytes) -> bool:
"""Verify that ``signature`` is the cryptographic signature of the hash
of ``data``."""
...
def _serialize_to_protobuf(self) -> protobuf.PublicKey:
"""Return the protobuf representation of this ``Key``."""
key_type = self.get_type().value
data = self.to_bytes()
protobuf_key = protobuf.PublicKey(key_type=key_type, data=data)
return protobuf_key
def serialize(self) -> bytes:
"""Return the canonical serialization of this ``Key``."""
return self._serialize_to_protobuf().SerializeToString()
@classmethod
def deserialize_from_protobuf(cls, protobuf_data: bytes) -> protobuf.PublicKey:
return protobuf.PublicKey.FromString(protobuf_data)
class PrivateKey(Key):
"""A ``PrivateKey`` represents a cryptographic private key."""
@abstractmethod
def sign(self, data: bytes) -> bytes:
...
@abstractmethod
def get_public_key(self) -> PublicKey:
...
def _serialize_to_protobuf(self) -> protobuf.PrivateKey:
"""Return the protobuf representation of this ``Key``."""
key_type = self.get_type().value
data = self.to_bytes()
protobuf_key = protobuf.PrivateKey(key_type=key_type, data=data)
return protobuf_key
def serialize(self) -> bytes:
"""Return the canonical serialization of this ``Key``."""
return self._serialize_to_protobuf().SerializeToString()
@classmethod
def deserialize_from_protobuf(cls, protobuf_data: bytes) -> protobuf.PrivateKey:
return protobuf.PrivateKey.FromString(protobuf_data)
@dataclass(frozen=True)
class KeyPair:
private_key: PrivateKey
public_key: PublicKey

View File

@ -0,0 +1,20 @@
syntax = "proto2";
package crypto.pb;
enum KeyType {
RSA = 0;
Ed25519 = 1;
Secp256k1 = 2;
ECDSA = 3;
}
message PublicKey {
required KeyType key_type = 1;
required bytes data = 2;
}
message PrivateKey {
required KeyType key_type = 1;
required bytes data = 2;
}

View File

@ -0,0 +1,162 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: libp2p/crypto/pb/crypto.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf.internal import enum_type_wrapper
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='libp2p/crypto/pb/crypto.proto',
package='crypto.pb',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x1dlibp2p/crypto/pb/crypto.proto\x12\tcrypto.pb\"?\n\tPublicKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c\"@\n\nPrivateKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c*9\n\x07KeyType\x12\x07\n\x03RSA\x10\x00\x12\x0b\n\x07\x45\x64\x32\x35\x35\x31\x39\x10\x01\x12\r\n\tSecp256k1\x10\x02\x12\t\n\x05\x45\x43\x44SA\x10\x03')
)
_KEYTYPE = _descriptor.EnumDescriptor(
name='KeyType',
full_name='crypto.pb.KeyType',
filename=None,
file=DESCRIPTOR,
values=[
_descriptor.EnumValueDescriptor(
name='RSA', index=0, number=0,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='Ed25519', index=1, number=1,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='Secp256k1', index=2, number=2,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='ECDSA', index=3, number=3,
serialized_options=None,
type=None),
],
containing_type=None,
serialized_options=None,
serialized_start=175,
serialized_end=232,
)
_sym_db.RegisterEnumDescriptor(_KEYTYPE)
KeyType = enum_type_wrapper.EnumTypeWrapper(_KEYTYPE)
RSA = 0
Ed25519 = 1
Secp256k1 = 2
ECDSA = 3
_PUBLICKEY = _descriptor.Descriptor(
name='PublicKey',
full_name='crypto.pb.PublicKey',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='key_type', full_name='crypto.pb.PublicKey.key_type', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='data', full_name='crypto.pb.PublicKey.data', index=1,
number=2, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto2',
extension_ranges=[],
oneofs=[
],
serialized_start=44,
serialized_end=107,
)
_PRIVATEKEY = _descriptor.Descriptor(
name='PrivateKey',
full_name='crypto.pb.PrivateKey',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='key_type', full_name='crypto.pb.PrivateKey.key_type', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='data', full_name='crypto.pb.PrivateKey.data', index=1,
number=2, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto2',
extension_ranges=[],
oneofs=[
],
serialized_start=109,
serialized_end=173,
)
_PUBLICKEY.fields_by_name['key_type'].enum_type = _KEYTYPE
_PRIVATEKEY.fields_by_name['key_type'].enum_type = _KEYTYPE
DESCRIPTOR.message_types_by_name['PublicKey'] = _PUBLICKEY
DESCRIPTOR.message_types_by_name['PrivateKey'] = _PRIVATEKEY
DESCRIPTOR.enum_types_by_name['KeyType'] = _KEYTYPE
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
PublicKey = _reflection.GeneratedProtocolMessageType('PublicKey', (_message.Message,), {
'DESCRIPTOR' : _PUBLICKEY,
'__module__' : 'libp2p.crypto.pb.crypto_pb2'
# @@protoc_insertion_point(class_scope:crypto.pb.PublicKey)
})
_sym_db.RegisterMessage(PublicKey)
PrivateKey = _reflection.GeneratedProtocolMessageType('PrivateKey', (_message.Message,), {
'DESCRIPTOR' : _PRIVATEKEY,
'__module__' : 'libp2p.crypto.pb.crypto_pb2'
# @@protoc_insertion_point(class_scope:crypto.pb.PrivateKey)
})
_sym_db.RegisterMessage(PrivateKey)
# @@protoc_insertion_point(module_scope)

View File

@ -0,0 +1,84 @@
# @generated by generate_proto_mypy_stubs.py. Do not edit!
import sys
from google.protobuf.descriptor import (
Descriptor as google___protobuf___descriptor___Descriptor,
EnumDescriptor as google___protobuf___descriptor___EnumDescriptor,
)
from google.protobuf.message import (
Message as google___protobuf___message___Message,
)
from typing import (
List as typing___List,
Tuple as typing___Tuple,
cast as typing___cast,
)
from typing_extensions import (
Literal as typing_extensions___Literal,
)
class KeyType(int):
DESCRIPTOR: google___protobuf___descriptor___EnumDescriptor = ...
@classmethod
def Name(cls, number: int) -> str: ...
@classmethod
def Value(cls, name: str) -> KeyType: ...
@classmethod
def keys(cls) -> typing___List[str]: ...
@classmethod
def values(cls) -> typing___List[KeyType]: ...
@classmethod
def items(cls) -> typing___List[typing___Tuple[str, KeyType]]: ...
RSA = typing___cast(KeyType, 0)
Ed25519 = typing___cast(KeyType, 1)
Secp256k1 = typing___cast(KeyType, 2)
ECDSA = typing___cast(KeyType, 3)
RSA = typing___cast(KeyType, 0)
Ed25519 = typing___cast(KeyType, 1)
Secp256k1 = typing___cast(KeyType, 2)
ECDSA = typing___cast(KeyType, 3)
class PublicKey(google___protobuf___message___Message):
DESCRIPTOR: google___protobuf___descriptor___Descriptor = ...
key_type = ... # type: KeyType
data = ... # type: bytes
def __init__(self,
*,
key_type : KeyType,
data : bytes,
) -> None: ...
@classmethod
def FromString(cls, s: bytes) -> PublicKey: ...
def MergeFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
def CopyFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
if sys.version_info >= (3,):
def HasField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> None: ...
else:
def HasField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> None: ...
class PrivateKey(google___protobuf___message___Message):
DESCRIPTOR: google___protobuf___descriptor___Descriptor = ...
key_type = ... # type: KeyType
data = ... # type: bytes
def __init__(self,
*,
key_type : KeyType,
data : bytes,
) -> None: ...
@classmethod
def FromString(cls, s: bytes) -> PrivateKey: ...
def MergeFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
def CopyFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
if sys.version_info >= (3,):
def HasField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> None: ...
else:
def HasField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> None: ...

65
libp2p/crypto/rsa.py Normal file
View File

@ -0,0 +1,65 @@
from Crypto.Hash import SHA256
import Crypto.PublicKey.RSA as RSA
from Crypto.PublicKey.RSA import RsaKey
from Crypto.Signature import pkcs1_15
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
class RSAPublicKey(PublicKey):
def __init__(self, impl: RsaKey) -> None:
self.impl = impl
def to_bytes(self) -> bytes:
return self.impl.export_key("DER")
@classmethod
def from_bytes(cls, key_bytes: bytes) -> "RSAPublicKey":
rsakey = RSA.import_key(key_bytes)
return cls(rsakey)
def get_type(self) -> KeyType:
return KeyType.RSA
def verify(self, data: bytes, signature: bytes) -> bool:
h = SHA256.new(data)
try:
pkcs1_15.new(self.impl).verify(h, signature)
except (ValueError, TypeError):
return False
return True
class RSAPrivateKey(PrivateKey):
def __init__(self, impl: RsaKey) -> None:
self.impl = impl
@classmethod
def new(cls, bits: int = 2048, e: int = 65537) -> "RSAPrivateKey":
private_key_impl = RSA.generate(bits, e=e)
return cls(private_key_impl)
def to_bytes(self) -> bytes:
return self.impl.export_key("DER")
def get_type(self) -> KeyType:
return KeyType.RSA
def sign(self, data: bytes) -> bytes:
h = SHA256.new(data)
return pkcs1_15.new(self.impl).sign(h)
def get_public_key(self) -> PublicKey:
return RSAPublicKey(self.impl.publickey())
def create_new_key_pair(bits: int = 2048, e: int = 65537) -> KeyPair:
"""
Returns a new RSA keypair with the requested key size (``bits``) and the
given public exponent ``e``.
Sane defaults are provided for both values.
"""
private_key = RSAPrivateKey.new(bits, e)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

View File

@ -0,0 +1,73 @@
import coincurve
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
class Secp256k1PublicKey(PublicKey):
def __init__(self, impl: coincurve.PublicKey) -> None:
self.impl = impl
def to_bytes(self) -> bytes:
return self.impl.format()
@classmethod
def from_bytes(cls, data: bytes) -> "Secp256k1PublicKey":
impl = coincurve.PublicKey(data)
return cls(impl)
@classmethod
def deserialize(cls, data: bytes) -> "Secp256k1PublicKey":
protobuf_key = cls.deserialize_from_protobuf(data)
return cls.from_bytes(protobuf_key.data)
def get_type(self) -> KeyType:
return KeyType.Secp256k1
def verify(self, data: bytes, signature: bytes) -> bool:
return self.impl.verify(signature, data)
class Secp256k1PrivateKey(PrivateKey):
def __init__(self, impl: coincurve.PrivateKey) -> None:
self.impl = impl
@classmethod
def new(cls, secret: bytes = None) -> "Secp256k1PrivateKey":
private_key_impl = coincurve.PrivateKey(secret)
return cls(private_key_impl)
def to_bytes(self) -> bytes:
return self.impl.secret
@classmethod
def from_bytes(cls, data: bytes) -> "Secp256k1PrivateKey":
impl = coincurve.PrivateKey(data)
return cls(impl)
@classmethod
def deserialize(cls, data: bytes) -> "Secp256k1PrivateKey":
protobuf_key = cls.deserialize_from_protobuf(data)
return cls.from_bytes(protobuf_key.data)
def get_type(self) -> KeyType:
return KeyType.Secp256k1
def sign(self, data: bytes) -> bytes:
return self.impl.sign(data)
def get_public_key(self) -> PublicKey:
public_key_impl = coincurve.PublicKey.from_secret(self.impl.secret)
return Secp256k1PublicKey(public_key_impl)
def create_new_key_pair(secret: bytes = None) -> KeyPair:
"""
Returns a new Secp256k1 keypair derived from the provided ``secret``, a
sequence of bytes corresponding to some integer between 0 and the group
order.
A valid secret is created if ``None`` is passed.
"""
private_key = Secp256k1PrivateKey.new(secret)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

View File

@ -0,0 +1,38 @@
from libp2p.crypto.ed25519 import Ed25519PrivateKey, Ed25519PublicKey
from libp2p.crypto.exceptions import MissingDeserializerError
from libp2p.crypto.keys import KeyType, PrivateKey, PublicKey
from libp2p.crypto.rsa import RSAPublicKey
from libp2p.crypto.secp256k1 import Secp256k1PrivateKey, Secp256k1PublicKey
key_type_to_public_key_deserializer = {
KeyType.Secp256k1.value: Secp256k1PublicKey.from_bytes,
KeyType.RSA.value: RSAPublicKey.from_bytes,
KeyType.Ed25519.value: Ed25519PublicKey.from_bytes,
}
key_type_to_private_key_deserializer = {
KeyType.Secp256k1.value: Secp256k1PrivateKey.from_bytes,
KeyType.Ed25519.value: Ed25519PrivateKey.from_bytes,
}
def deserialize_public_key(data: bytes) -> PublicKey:
f = PublicKey.deserialize_from_protobuf(data)
try:
deserializer = key_type_to_public_key_deserializer[f.key_type]
except KeyError as e:
raise MissingDeserializerError(
{"key_type": f.key_type, "key": "public_key"}
) from e
return deserializer(f.data)
def deserialize_private_key(data: bytes) -> PrivateKey:
f = PrivateKey.deserialize_from_protobuf(data)
try:
deserializer = key_type_to_private_key_deserializer[f.key_type]
except KeyError as e:
raise MissingDeserializerError(
{"key_type": f.key_type, "key": "private_key"}
) from e
return deserializer(f.data)

16
libp2p/exceptions.py Normal file
View File

@ -0,0 +1,16 @@
class BaseLibp2pError(Exception):
pass
class ValidationError(BaseLibp2pError):
"""Raised when something does not pass a validation check."""
class ParseError(BaseLibp2pError):
pass
class MultiError(BaseLibp2pError):
"""Raised with multiple exceptions."""
# todo: find some way for this to fancy-print all encapsulated errors

View File

@ -1,90 +1,195 @@
import logging
from typing import TYPE_CHECKING, AsyncIterator, List, Sequence
from async_generator import asynccontextmanager
from async_service import background_trio_service
import multiaddr
from libp2p.crypto.keys import PrivateKey, PublicKey
from libp2p.host.defaults import get_default_protocols
from libp2p.host.exceptions import StreamFailure
from libp2p.network.network_interface import INetworkService
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import PeerInfo
from libp2p.peer.peerstore_interface import IPeerStore
from libp2p.protocol_muxer.exceptions import MultiselectClientError, MultiselectError
from libp2p.protocol_muxer.multiselect import Multiselect
from libp2p.protocol_muxer.multiselect_client import MultiselectClient
from libp2p.protocol_muxer.multiselect_communicator import MultiselectCommunicator
from libp2p.typing import StreamHandlerFn, TProtocol
from .host_interface import IHost
if TYPE_CHECKING:
from collections import OrderedDict
# Upon host creation, host takes in options,
# including the list of addresses on which to listen.
# Host then parses these options and delegates to its Network instance,
# telling it to listen on the given listen addresses.
logger = logging.getLogger("libp2p.network.basic_host")
class BasicHost(IHost):
"""
BasicHost is a wrapper of a `INetwork` implementation.
# default options constructor
def __init__(self, _network):
self.network = _network
self.peerstore = self.network.peerstore
It performs protocol negotiation on a stream with multistream-select
right after a stream is initialized.
"""
def get_id(self):
_network: INetworkService
peerstore: IPeerStore
multiselect: Multiselect
multiselect_client: MultiselectClient
def __init__(
self,
network: INetworkService,
default_protocols: "OrderedDict[TProtocol, StreamHandlerFn]" = None,
) -> None:
self._network = network
self._network.set_stream_handler(self._swarm_stream_handler)
self.peerstore = self._network.peerstore
# Protocol muxing
default_protocols = default_protocols or get_default_protocols(self)
self.multiselect = Multiselect(default_protocols)
self.multiselect_client = MultiselectClient()
def get_id(self) -> ID:
"""
:return: peer_id of host
"""
return self.network.get_peer_id()
return self._network.get_peer_id()
def get_network(self):
def get_public_key(self) -> PublicKey:
return self.peerstore.pubkey(self.get_id())
def get_private_key(self) -> PrivateKey:
return self.peerstore.privkey(self.get_id())
def get_network(self) -> INetworkService:
"""
:return: network instance of host
"""
return self.network
return self._network
def get_peerstore(self):
def get_peerstore(self) -> IPeerStore:
"""
:return: peerstore of the host (same one as in its network instance)
"""
return self.peerstore
def get_mux(self):
def get_mux(self) -> Multiselect:
"""
:return: mux instance of host
"""
return self.multiselect
def get_addrs(self):
def get_addrs(self) -> List[multiaddr.Multiaddr]:
"""
:return: all the multiaddr addresses this host is listening too
:return: all the multiaddr addresses this host is listening to
"""
p2p_part = multiaddr.Multiaddr('/p2p/{}'.format(self.get_id().pretty()))
# TODO: We don't need "/p2p/{peer_id}" postfix actually.
p2p_part = multiaddr.Multiaddr(f"/p2p/{self.get_id()!s}")
addrs = []
for transport in self.network.listeners.values():
addrs: List[multiaddr.Multiaddr] = []
for transport in self._network.listeners.values():
for addr in transport.get_addrs():
addrs.append(addr.encapsulate(p2p_part))
return addrs
def set_stream_handler(self, protocol_id, stream_handler):
@asynccontextmanager
async def run(
self, listen_addrs: Sequence[multiaddr.Multiaddr]
) -> AsyncIterator[None]:
"""
set stream handler for host
run the host instance and listen to ``listen_addrs``.
:param listen_addrs: a sequence of multiaddrs that we want to listen to
"""
network = self.get_network()
async with background_trio_service(network):
await network.listen(*listen_addrs)
yield
def set_stream_handler(
self, protocol_id: TProtocol, stream_handler: StreamHandlerFn
) -> None:
"""
set stream handler for given `protocol_id`
:param protocol_id: protocol id used on stream
:param stream_handler: a stream handler function
:return: true if successful
"""
return self.network.set_stream_handler(protocol_id, stream_handler)
self.multiselect.add_handler(protocol_id, stream_handler)
# protocol_id can be a list of protocol_ids
# stream will decide which protocol_id to run on
async def new_stream(self, peer_id, protocol_ids):
async def new_stream(
self, peer_id: ID, protocol_ids: Sequence[TProtocol]
) -> INetStream:
"""
:param peer_id: peer_id that host is connecting
:param protocol_id: protocol id that stream runs on
:return: true if successful
:param protocol_ids: available protocol ids to use for stream
:return: stream: new stream created
"""
stream = await self.network.new_stream(peer_id, protocol_ids)
return stream
async def connect(self, peer_info):
net_stream = await self._network.new_stream(peer_id)
# Perform protocol muxing to determine protocol to use
try:
selected_protocol = await self.multiselect_client.select_one_of(
list(protocol_ids), MultiselectCommunicator(net_stream)
)
except MultiselectClientError as error:
logger.debug("fail to open a stream to peer %s, error=%s", peer_id, error)
await net_stream.reset()
raise StreamFailure(f"failed to open a stream to peer {peer_id}") from error
net_stream.set_protocol(selected_protocol)
return net_stream
async def connect(self, peer_info: PeerInfo) -> None:
"""
connect ensures there is a connection between this host and the peer with
given peer_info.peer_id. connect will absorb the addresses in peer_info into its internal
peerstore. If there is not an active connection, connect will issue a
dial, and block until a connection is open, or an error is
returned.
connect ensures there is a connection between this host and the peer
with given `peer_info.peer_id`. connect will absorb the addresses in
peer_info into its internal peerstore. If there is not an active
connection, connect will issue a dial, and block until a connection is
opened, or an error is returned.
:param peer_info: peer_info of the host we want to connect to
:param peer_info: peer_info of the peer we want to connect to
:type peer_info: peer.peerinfo.PeerInfo
"""
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
# there is already a connection to this peer
if peer_info.peer_id in self.network.connections:
if peer_info.peer_id in self._network.connections:
return
await self.network.dial_peer(peer_info.peer_id)
await self._network.dial_peer(peer_info.peer_id)
async def disconnect(self, peer_id: ID) -> None:
await self._network.close_peer(peer_id)
async def close(self) -> None:
await self._network.close()
# Reference: `BasicHost.newStreamHandler` in Go.
async def _swarm_stream_handler(self, net_stream: INetStream) -> None:
# Perform protocol muxing to determine protocol to use
try:
protocol, handler = await self.multiselect.negotiate(
MultiselectCommunicator(net_stream)
)
except MultiselectError as error:
peer_id = net_stream.muxed_conn.peer_id
logger.debug(
"failed to accept a stream from peer %s, error=%s", peer_id, error
)
await net_stream.reset()
return
net_stream.set_protocol(protocol)
await handler(net_stream)

17
libp2p/host/defaults.py Normal file
View File

@ -0,0 +1,17 @@
from collections import OrderedDict
from typing import TYPE_CHECKING
from libp2p.host.host_interface import IHost
from libp2p.host.ping import ID as PingID
from libp2p.host.ping import handle_ping
from libp2p.identity.identify.protocol import ID as IdentifyID
from libp2p.identity.identify.protocol import identify_handler_for
if TYPE_CHECKING:
from libp2p.typing import TProtocol, StreamHandlerFn
def get_default_protocols(host: IHost) -> "OrderedDict[TProtocol, StreamHandlerFn]":
return OrderedDict(
((IdentifyID, identify_handler_for(host)), (PingID, handle_ping))
)

13
libp2p/host/exceptions.py Normal file
View File

@ -0,0 +1,13 @@
from libp2p.exceptions import BaseLibp2pError
class HostException(BaseLibp2pError):
"""A generic exception in `IHost`."""
class ConnectionFailure(HostException):
pass
class StreamFailure(HostException):
pass

View File

@ -1,60 +1,104 @@
from abc import ABC, abstractmethod
from typing import Any, AsyncContextManager, List, Sequence
import multiaddr
from libp2p.crypto.keys import PrivateKey, PublicKey
from libp2p.network.network_interface import INetworkService
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import PeerInfo
from libp2p.typing import StreamHandlerFn, TProtocol
class IHost(ABC):
@abstractmethod
def get_id(self):
def get_id(self) -> ID:
"""
:return: peer_id of host
"""
@abstractmethod
def get_network(self):
def get_public_key(self) -> PublicKey:
"""
:return: the public key belonging to the peer
"""
@abstractmethod
def get_private_key(self) -> PrivateKey:
"""
:return: the private key belonging to the peer
"""
@abstractmethod
def get_network(self) -> INetworkService:
"""
:return: network instance of host
"""
# FIXME: Replace with correct return type
@abstractmethod
def get_mux(self):
def get_mux(self) -> Any:
"""
:return: mux instance of host
"""
@abstractmethod
def get_addrs(self):
def get_addrs(self) -> List[multiaddr.Multiaddr]:
"""
:return: all the multiaddr addresses this host is listening too
:return: all the multiaddr addresses this host is listening to
"""
@abstractmethod
def set_stream_handler(self, protocol_id, stream_handler):
def run(
self, listen_addrs: Sequence[multiaddr.Multiaddr]
) -> AsyncContextManager[None]:
"""
set stream handler for host
run the host instance and listen to ``listen_addrs``.
:param listen_addrs: a sequence of multiaddrs that we want to listen to
"""
@abstractmethod
def set_stream_handler(
self, protocol_id: TProtocol, stream_handler: StreamHandlerFn
) -> None:
"""
set stream handler for host.
:param protocol_id: protocol id used on stream
:param stream_handler: a stream handler function
:return: true if successful
"""
# protocol_id can be a list of protocol_ids
# stream will decide which protocol_id to run on
@abstractmethod
def new_stream(self, peer_id, protocol_ids):
async def new_stream(
self, peer_id: ID, protocol_ids: Sequence[TProtocol]
) -> INetStream:
"""
:param peer_id: peer_id that host is connecting
:param protocol_ids: protocol ids that stream can run on
:return: true if successful
:param protocol_ids: available protocol ids to use for stream
:return: stream: new stream created
"""
@abstractmethod
def connect(self, peer_info):
async def connect(self, peer_info: PeerInfo) -> None:
"""
connect ensures there is a connection between this host and the peer with
given peer_info.peer_id. connect will absorb the addresses in peer_info into its internal
peerstore. If there is not an active connection, connect will issue a
dial, and block until a connection is open, or an error is
returned.
connect ensures there is a connection between this host and the peer
with given peer_info.peer_id. connect will absorb the addresses in
peer_info into its internal peerstore. If there is not an active
connection, connect will issue a dial, and block until a connection is
opened, or an error is returned.
:param peer_info: peer_info of the host we want to connect to
:param peer_info: peer_info of the peer we want to connect to
:type peer_info: peer.peerinfo.PeerInfo
"""
@abstractmethod
async def disconnect(self, peer_id: ID) -> None:
pass
@abstractmethod
async def close(self) -> None:
pass

60
libp2p/host/ping.py Normal file
View File

@ -0,0 +1,60 @@
import logging
import trio
from libp2p.network.stream.exceptions import StreamClosed, StreamEOF, StreamReset
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.id import ID as PeerID
from libp2p.typing import TProtocol
ID = TProtocol("/ipfs/ping/1.0.0")
PING_LENGTH = 32
RESP_TIMEOUT = 60
logger = logging.getLogger("libp2p.host.ping")
async def _handle_ping(stream: INetStream, peer_id: PeerID) -> bool:
"""Return a boolean indicating if we expect more pings from the peer at
``peer_id``."""
try:
with trio.fail_after(RESP_TIMEOUT):
payload = await stream.read(PING_LENGTH)
except trio.TooSlowError as error:
logger.debug("Timed out waiting for ping from %s: %s", peer_id, error)
raise
except StreamEOF:
logger.debug("Other side closed while waiting for ping from %s", peer_id)
return False
except StreamReset as error:
logger.debug(
"Other side reset while waiting for ping from %s: %s", peer_id, error
)
raise
except Exception as error:
logger.debug("Error while waiting to read ping for %s: %s", peer_id, error)
raise
logger.debug("Received ping from %s with data: 0x%s", peer_id, payload.hex())
try:
await stream.write(payload)
except StreamClosed:
logger.debug("Fail to respond to ping from %s: stream closed", peer_id)
raise
return True
async def handle_ping(stream: INetStream) -> None:
"""``handle_ping`` responds to incoming ping requests until one side errors
or closes the ``stream``."""
peer_id = stream.muxed_conn.peer_id
while True:
try:
should_continue = await _handle_ping(stream, peer_id)
if not should_continue:
return
except Exception:
await stream.reset()
return

View File

@ -0,0 +1,41 @@
from libp2p.host.basic_host import BasicHost
from libp2p.host.exceptions import ConnectionFailure
from libp2p.network.network_interface import INetworkService
from libp2p.peer.peerinfo import PeerInfo
from libp2p.routing.interfaces import IPeerRouting
# RoutedHost is a p2p Host that includes a routing system.
# This allows the Host to find the addresses for peers when it does not have them.
class RoutedHost(BasicHost):
_router: IPeerRouting
def __init__(self, network: INetworkService, router: IPeerRouting):
super().__init__(network)
self._router = router
async def connect(self, peer_info: PeerInfo) -> None:
"""
connect ensures there is a connection between this host and the peer
with given `peer_info.peer_id`. See (basic_host).connect for more
information.
RoutedHost's Connect differs in that if the host has no addresses for a
given peer, it will use its routing system to try to find some.
:param peer_info: peer_info of the peer we want to connect to
:type peer_info: peer.peerinfo.PeerInfo
"""
# check if we were given some addresses, otherwise, find some with the routing system.
if not peer_info.addrs:
found_peer_info = await self._router.find_peer(peer_info.peer_id)
if not found_peer_info:
raise ConnectionFailure("Unable to find Peer address")
self.peerstore.add_addrs(peer_info.peer_id, found_peer_info.addrs, 10)
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
# there is already a connection to this peer
if peer_info.peer_id in self._network.connections:
return
await self._network.dial_peer(peer_info.peer_id)

View File

@ -0,0 +1,12 @@
syntax = "proto2";
package identify.pb;
message Identify {
optional string protocol_version = 5;
optional string agent_version = 6;
optional bytes public_key = 1;
repeated bytes listen_addrs = 2;
optional bytes observed_addr = 4;
repeated string protocols = 3;
}

View File

@ -0,0 +1,105 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: libp2p/identity/identify/pb/identify.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='libp2p/identity/identify/pb/identify.proto',
package='identify.pb',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n*libp2p/identity/identify/pb/identify.proto\x12\x0bidentify.pb\"\x8f\x01\n\x08Identify\x12\x18\n\x10protocol_version\x18\x05 \x01(\t\x12\x15\n\ragent_version\x18\x06 \x01(\t\x12\x12\n\npublic_key\x18\x01 \x01(\x0c\x12\x14\n\x0clisten_addrs\x18\x02 \x03(\x0c\x12\x15\n\robserved_addr\x18\x04 \x01(\x0c\x12\x11\n\tprotocols\x18\x03 \x03(\t')
)
_IDENTIFY = _descriptor.Descriptor(
name='Identify',
full_name='identify.pb.Identify',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='protocol_version', full_name='identify.pb.Identify.protocol_version', index=0,
number=5, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='agent_version', full_name='identify.pb.Identify.agent_version', index=1,
number=6, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='public_key', full_name='identify.pb.Identify.public_key', index=2,
number=1, type=12, cpp_type=9, label=1,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='listen_addrs', full_name='identify.pb.Identify.listen_addrs', index=3,
number=2, type=12, cpp_type=9, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='observed_addr', full_name='identify.pb.Identify.observed_addr', index=4,
number=4, type=12, cpp_type=9, label=1,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='protocols', full_name='identify.pb.Identify.protocols', index=5,
number=3, type=9, cpp_type=9, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto2',
extension_ranges=[],
oneofs=[
],
serialized_start=60,
serialized_end=203,
)
DESCRIPTOR.message_types_by_name['Identify'] = _IDENTIFY
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
Identify = _reflection.GeneratedProtocolMessageType('Identify', (_message.Message,), {
'DESCRIPTOR' : _IDENTIFY,
'__module__' : 'libp2p.identity.identify.pb.identify_pb2'
# @@protoc_insertion_point(class_scope:identify.pb.Identify)
})
_sym_db.RegisterMessage(Identify)
# @@protoc_insertion_point(module_scope)

View File

@ -0,0 +1,53 @@
# @generated by generate_proto_mypy_stubs.py. Do not edit!
import sys
from google.protobuf.descriptor import (
Descriptor as google___protobuf___descriptor___Descriptor,
)
from google.protobuf.internal.containers import (
RepeatedScalarFieldContainer as google___protobuf___internal___containers___RepeatedScalarFieldContainer,
)
from google.protobuf.message import (
Message as google___protobuf___message___Message,
)
from typing import (
Iterable as typing___Iterable,
Optional as typing___Optional,
Text as typing___Text,
)
from typing_extensions import (
Literal as typing_extensions___Literal,
)
class Identify(google___protobuf___message___Message):
DESCRIPTOR: google___protobuf___descriptor___Descriptor = ...
protocol_version = ... # type: typing___Text
agent_version = ... # type: typing___Text
public_key = ... # type: bytes
listen_addrs = ... # type: google___protobuf___internal___containers___RepeatedScalarFieldContainer[bytes]
observed_addr = ... # type: bytes
protocols = ... # type: google___protobuf___internal___containers___RepeatedScalarFieldContainer[typing___Text]
def __init__(self,
*,
protocol_version : typing___Optional[typing___Text] = None,
agent_version : typing___Optional[typing___Text] = None,
public_key : typing___Optional[bytes] = None,
listen_addrs : typing___Optional[typing___Iterable[bytes]] = None,
observed_addr : typing___Optional[bytes] = None,
protocols : typing___Optional[typing___Iterable[typing___Text]] = None,
) -> None: ...
@classmethod
def FromString(cls, s: bytes) -> Identify: ...
def MergeFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
def CopyFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
if sys.version_info >= (3,):
def HasField(self, field_name: typing_extensions___Literal[u"agent_version",u"observed_addr",u"protocol_version",u"public_key"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"agent_version",u"listen_addrs",u"observed_addr",u"protocol_version",u"protocols",u"public_key"]) -> None: ...
else:
def HasField(self, field_name: typing_extensions___Literal[u"agent_version",b"agent_version",u"observed_addr",b"observed_addr",u"protocol_version",b"protocol_version",u"public_key",b"public_key"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"agent_version",b"agent_version",u"listen_addrs",b"listen_addrs",u"observed_addr",b"observed_addr",u"protocol_version",b"protocol_version",u"protocols",b"protocols",u"public_key",b"public_key"]) -> None: ...

View File

@ -0,0 +1,55 @@
import logging
from multiaddr import Multiaddr
from libp2p.host.host_interface import IHost
from libp2p.network.stream.exceptions import StreamClosed
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.typing import StreamHandlerFn, TProtocol
from .pb.identify_pb2 import Identify
ID = TProtocol("/ipfs/id/1.0.0")
PROTOCOL_VERSION = "ipfs/0.1.0"
# TODO dynamically generate the agent version
AGENT_VERSION = "py-libp2p/alpha"
logger = logging.getLogger("libp2p.identity.identify")
def _multiaddr_to_bytes(maddr: Multiaddr) -> bytes:
return maddr.to_bytes()
def _mk_identify_protobuf(host: IHost) -> Identify:
public_key = host.get_public_key()
laddrs = host.get_addrs()
protocols = host.get_mux().get_protocols()
return Identify(
protocol_version=PROTOCOL_VERSION,
agent_version=AGENT_VERSION,
public_key=public_key.serialize(),
listen_addrs=map(_multiaddr_to_bytes, laddrs),
# TODO send observed address from ``stream``
observed_addr=b"",
protocols=protocols,
)
def identify_handler_for(host: IHost) -> StreamHandlerFn:
async def handle_identify(stream: INetStream) -> None:
peer_id = stream.muxed_conn.peer_id
logger.debug("received a request for %s from %s", ID, peer_id)
protobuf = _mk_identify_protobuf(host)
response = protobuf.SerializeToString()
try:
await stream.write(response)
except StreamClosed:
logger.debug("Fail to respond to %s request: stream closed", ID)
else:
await stream.close()
logger.debug("successfully handled request for %s from %s", ID, peer_id)
return handle_identify

0
libp2p/io/__init__.py Normal file
View File

65
libp2p/io/abc.py Normal file
View File

@ -0,0 +1,65 @@
from abc import ABC, abstractmethod
class Closer(ABC):
@abstractmethod
async def close(self) -> None:
...
class Reader(ABC):
@abstractmethod
async def read(self, n: int = None) -> bytes:
...
class Writer(ABC):
@abstractmethod
async def write(self, data: bytes) -> None:
...
class WriteCloser(Writer, Closer):
pass
class ReadCloser(Reader, Closer):
pass
class ReadWriter(Reader, Writer):
pass
class ReadWriteCloser(Reader, Writer, Closer):
pass
class MsgReader(ABC):
@abstractmethod
async def read_msg(self) -> bytes:
...
class MsgWriter(ABC):
@abstractmethod
async def write_msg(self, msg: bytes) -> None:
...
class MsgReadWriteCloser(MsgReader, MsgWriter, Closer):
pass
class Encrypter(ABC):
@abstractmethod
def encrypt(self, data: bytes) -> bytes:
...
@abstractmethod
def decrypt(self, data: bytes) -> bytes:
...
class EncryptedMsgReadWriter(MsgReadWriteCloser, Encrypter):
"""Read/write message with encryption/decryption."""

29
libp2p/io/exceptions.py Normal file
View File

@ -0,0 +1,29 @@
from libp2p.exceptions import BaseLibp2pError
class IOException(BaseLibp2pError):
pass
class IncompleteReadError(IOException):
"""Fewer bytes were read than requested."""
class MsgioException(IOException):
pass
class MissingLengthException(MsgioException):
pass
class MissingMessageException(MsgioException):
pass
class DecryptionFailedException(MsgioException):
pass
class MessageTooLarge(MsgioException):
pass

89
libp2p/io/msgio.py Normal file
View File

@ -0,0 +1,89 @@
"""
``msgio`` is an implementation of `https://github.com/libp2p/go-msgio`.
from that repo: "a simple package to r/w length-delimited slices."
NOTE: currently missing the capability to indicate lengths by "varint" method.
"""
from abc import abstractmethod
from libp2p.io.abc import MsgReadWriteCloser, Reader, ReadWriteCloser
from libp2p.io.utils import read_exactly
from libp2p.utils import decode_uvarint_from_stream, encode_varint_prefixed
from .exceptions import MessageTooLarge
BYTE_ORDER = "big"
async def read_length(reader: Reader, size_len_bytes: int) -> int:
length_bytes = await read_exactly(reader, size_len_bytes)
return int.from_bytes(length_bytes, byteorder=BYTE_ORDER)
def encode_msg_with_length(msg_bytes: bytes, size_len_bytes: int) -> bytes:
try:
len_prefix = len(msg_bytes).to_bytes(size_len_bytes, byteorder=BYTE_ORDER)
except OverflowError:
raise ValueError(
"msg_bytes is too large for `size_len_bytes` bytes length: "
f"msg_bytes={msg_bytes!r}, size_len_bytes={size_len_bytes}"
)
return len_prefix + msg_bytes
class BaseMsgReadWriter(MsgReadWriteCloser):
read_write_closer: ReadWriteCloser
size_len_bytes: int
def __init__(self, read_write_closer: ReadWriteCloser) -> None:
self.read_write_closer = read_write_closer
async def read_msg(self) -> bytes:
length = await self.next_msg_len()
return await read_exactly(self.read_write_closer, length)
@abstractmethod
async def next_msg_len(self) -> int:
...
@abstractmethod
def encode_msg(self, msg: bytes) -> bytes:
...
async def close(self) -> None:
await self.read_write_closer.close()
async def write_msg(self, msg: bytes) -> None:
encoded_msg = self.encode_msg(msg)
await self.read_write_closer.write(encoded_msg)
class FixedSizeLenMsgReadWriter(BaseMsgReadWriter):
size_len_bytes: int
async def next_msg_len(self) -> int:
return await read_length(self.read_write_closer, self.size_len_bytes)
def encode_msg(self, msg: bytes) -> bytes:
return encode_msg_with_length(msg, self.size_len_bytes)
class VarIntLengthMsgReadWriter(BaseMsgReadWriter):
max_msg_size: int
async def next_msg_len(self) -> int:
msg_len = await decode_uvarint_from_stream(self.read_write_closer)
if msg_len > self.max_msg_size:
raise MessageTooLarge(
f"msg_len={msg_len} > max_msg_size={self.max_msg_size}"
)
return msg_len
def encode_msg(self, msg: bytes) -> bytes:
msg_len = len(msg)
if msg_len > self.max_msg_size:
raise MessageTooLarge(
f"msg_len={msg_len} > max_msg_size={self.max_msg_size}"
)
return encode_varint_prefixed(msg)

40
libp2p/io/trio.py Normal file
View File

@ -0,0 +1,40 @@
import logging
import trio
from libp2p.io.abc import ReadWriteCloser
from libp2p.io.exceptions import IOException
logger = logging.getLogger("libp2p.io.trio")
class TrioTCPStream(ReadWriteCloser):
stream: trio.SocketStream
# NOTE: Add both read and write lock to avoid `trio.BusyResourceError`
read_lock: trio.Lock
write_lock: trio.Lock
def __init__(self, stream: trio.SocketStream) -> None:
self.stream = stream
self.read_lock = trio.Lock()
self.write_lock = trio.Lock()
async def write(self, data: bytes) -> None:
"""Raise `RawConnError` if the underlying connection breaks."""
async with self.write_lock:
try:
await self.stream.send_all(data)
except (trio.ClosedResourceError, trio.BrokenResourceError) as error:
raise IOException from error
async def read(self, n: int = None) -> bytes:
async with self.read_lock:
if n is not None and n == 0:
return b""
try:
return await self.stream.receive_some(n)
except (trio.ClosedResourceError, trio.BrokenResourceError) as error:
raise IOException from error
async def close(self) -> None:
await self.stream.aclose()

21
libp2p/io/utils.py Normal file
View File

@ -0,0 +1,21 @@
from libp2p.io.abc import Reader
from libp2p.io.exceptions import IncompleteReadError
DEFAULT_RETRY_READ_COUNT = 100
async def read_exactly(
reader: Reader, n: int, retry_count: int = DEFAULT_RETRY_READ_COUNT
) -> bytes:
"""
NOTE: relying on exceptions to break out on erroneous conditions, like EOF
"""
data = await reader.read(n)
for _ in range(retry_count):
if len(data) < n:
remaining = n - len(data)
data += await reader.read(remaining)
else:
return data
raise IncompleteReadError({"requested_count": n, "received_count": len(data)})

View File

@ -1,5 +0,0 @@
"""
Kademlia is a Python implementation of the Kademlia protocol which
utilizes the asyncio library.
"""
__version__ = "1.1"

View File

@ -1,181 +0,0 @@
from collections import Counter
import logging
from .kademlia.node import Node, NodeHeap
from .kademlia.utils import gather_dict
log = logging.getLogger(__name__)
class SpiderCrawl:
"""
Crawl the network and look for given 160-bit keys.
"""
def __init__(self, protocol, node, peers, ksize, alpha):
"""
Create a new C{SpiderCrawl}er.
Args:
protocol: A :class:`~kademlia.protocol.KademliaProtocol` instance.
node: A :class:`~kademlia.node.Node` representing the key we're
looking for
peers: A list of :class:`~kademlia.node.Node` instances that
provide the entry point for the network
ksize: The value for k based on the paper
alpha: The value for alpha based on the paper
"""
self.protocol = protocol
self.ksize = ksize
self.alpha = alpha
self.node = node
self.nearest = NodeHeap(self.node, self.ksize)
self.lastIDsCrawled = []
log.info("creating spider with peers: %s", peers)
self.nearest.push(peers)
async def _find(self, rpcmethod):
"""
Get either a value or list of nodes.
Args:
rpcmethod: The protocol's callfindValue or callFindNode.
The process:
1. calls find_* to current ALPHA nearest not already queried nodes,
adding results to current nearest list of k nodes.
2. current nearest list needs to keep track of who has been queried
already sort by nearest, keep KSIZE
3. if list is same as last time, next call should be to everyone not
yet queried
4. repeat, unless nearest list has all been queried, then ur done
"""
log.info("crawling network with nearest: %s", str(tuple(self.nearest)))
count = self.alpha
if self.nearest.getIDs() == self.lastIDsCrawled:
count = len(self.nearest)
self.lastIDsCrawled = self.nearest.getIDs()
ds = {}
for peer in self.nearest.getUncontacted()[:count]:
ds[peer.id] = rpcmethod(peer, self.node)
self.nearest.markContacted(peer)
found = await gather_dict(ds)
return await self._nodesFound(found)
async def _nodesFound(self, responses):
raise NotImplementedError
class ValueSpiderCrawl(SpiderCrawl):
def __init__(self, protocol, node, peers, ksize, alpha):
SpiderCrawl.__init__(self, protocol, node, peers, ksize, alpha)
# keep track of the single nearest node without value - per
# section 2.3 so we can set the key there if found
self.nearestWithoutValue = NodeHeap(self.node, 1)
async def find(self):
"""
Find either the closest nodes or the value requested.
"""
return await self._find(self.protocol.callFindValue)
async def _nodesFound(self, responses):
"""
Handle the result of an iteration in _find.
"""
toremove = []
foundValues = []
for peerid, response in responses.items():
response = RPCFindResponse(response)
if not response.happened():
toremove.append(peerid)
elif response.hasValue():
foundValues.append(response.getValue())
else:
peer = self.nearest.getNodeById(peerid)
self.nearestWithoutValue.push(peer)
self.nearest.push(response.getNodeList())
self.nearest.remove(toremove)
if len(foundValues) > 0:
return await self._handleFoundValues(foundValues)
if self.nearest.allBeenContacted():
# not found!
return None
return await self.find()
async def _handleFoundValues(self, values):
"""
We got some values! Exciting. But let's make sure
they're all the same or freak out a little bit. Also,
make sure we tell the nearest node that *didn't* have
the value to store it.
"""
valueCounts = Counter(values)
if len(valueCounts) != 1:
log.warning("Got multiple values for key %i: %s",
self.node.long_id, str(values))
value = valueCounts.most_common(1)[0][0]
peerToSaveTo = self.nearestWithoutValue.popleft()
if peerToSaveTo is not None:
await self.protocol.callStore(peerToSaveTo, self.node.id, value)
return value
class NodeSpiderCrawl(SpiderCrawl):
async def find(self):
"""
Find the closest nodes.
"""
return await self._find(self.protocol.callFindNode)
async def _nodesFound(self, responses):
"""
Handle the result of an iteration in _find.
"""
toremove = []
for peerid, response in responses.items():
response = RPCFindResponse(response)
if not response.happened():
toremove.append(peerid)
else:
self.nearest.push(response.getNodeList())
self.nearest.remove(toremove)
if self.nearest.allBeenContacted():
return list(self.nearest)
return await self.find()
class RPCFindResponse:
def __init__(self, response):
"""
A wrapper for the result of a RPC find.
Args:
response: This will be a tuple of (<response received>, <value>)
where <value> will be a list of tuples if not found or
a dictionary of {'value': v} where v is the value desired
"""
self.response = response
def happened(self):
"""
Did the other host actually respond?
"""
return self.response[0]
def hasValue(self):
return isinstance(self.response[1], dict)
def getValue(self):
return self.response[1]['value']
def getNodeList(self):
"""
Get the node list in the response. If there's no value, this should
be set.
"""
nodelist = self.response[1] or []
return [Node(*nodeple) for nodeple in nodelist]

View File

@ -1,258 +0,0 @@
"""
Package for interacting on the network at a high level.
"""
import random
import pickle
import asyncio
import logging
from .kademlia.protocol import KademliaProtocol
from .kademlia.utils import digest
from .kademlia.storage import ForgetfulStorage
from .kademlia.node import Node
from .kademlia.crawling import ValueSpiderCrawl
from .kademlia.crawling import NodeSpiderCrawl
log = logging.getLogger(__name__)
class Server:
"""
High level view of a node instance. This is the object that should be
created to start listening as an active node on the network.
"""
protocol_class = KademliaProtocol
def __init__(self, ksize=20, alpha=3, node_id=None, storage=None):
"""
Create a server instance. This will start listening on the given port.
Args:
ksize (int): The k parameter from the paper
alpha (int): The alpha parameter from the paper
node_id: The id for this node on the network.
storage: An instance that implements
:interface:`~kademlia.storage.IStorage`
"""
self.ksize = ksize
self.alpha = alpha
self.storage = storage or ForgetfulStorage()
self.node = Node(node_id or digest(random.getrandbits(255)))
self.transport = None
self.protocol = None
self.refresh_loop = None
self.save_state_loop = None
def stop(self):
if self.transport is not None:
self.transport.close()
if self.refresh_loop:
self.refresh_loop.cancel()
if self.save_state_loop:
self.save_state_loop.cancel()
def _create_protocol(self):
return self.protocol_class(self.node, self.storage, self.ksize)
def listen(self, port, interface='0.0.0.0'):
"""
Start listening on the given port.
Provide interface="::" to accept ipv6 address
"""
loop = asyncio.get_event_loop()
listen = loop.create_datagram_endpoint(self._create_protocol,
local_addr=(interface, port))
log.info("Node %i listening on %s:%i",
self.node.long_id, interface, port)
self.transport, self.protocol = loop.run_until_complete(listen)
# finally, schedule refreshing table
self.refresh_table()
def refresh_table(self):
log.debug("Refreshing routing table")
asyncio.ensure_future(self._refresh_table())
loop = asyncio.get_event_loop()
self.refresh_loop = loop.call_later(3600, self.refresh_table)
async def _refresh_table(self):
"""
Refresh buckets that haven't had any lookups in the last hour
(per section 2.3 of the paper).
"""
ds = []
for node_id in self.protocol.getRefreshIDs():
node = Node(node_id)
nearest = self.protocol.router.findNeighbors(node, self.alpha)
spider = NodeSpiderCrawl(self.protocol, node, nearest,
self.ksize, self.alpha)
ds.append(spider.find())
# do our crawling
await asyncio.gather(*ds)
# now republish keys older than one hour
for dkey, value in self.storage.iteritemsOlderThan(3600):
await self.set_digest(dkey, value)
def bootstrappableNeighbors(self):
"""
Get a :class:`list` of (ip, port) :class:`tuple` pairs suitable for
use as an argument to the bootstrap method.
The server should have been bootstrapped
already - this is just a utility for getting some neighbors and then
storing them if this server is going down for a while. When it comes
back up, the list of nodes can be used to bootstrap.
"""
neighbors = self.protocol.router.findNeighbors(self.node)
return [tuple(n)[-2:] for n in neighbors]
async def bootstrap(self, addrs):
"""
Bootstrap the server by connecting to other known nodes in the network.
Args:
addrs: A `list` of (ip, port) `tuple` pairs. Note that only IP
addresses are acceptable - hostnames will cause an error.
"""
log.debug("Attempting to bootstrap node with %i initial contacts",
len(addrs))
cos = list(map(self.bootstrap_node, addrs))
gathered = await asyncio.gather(*cos)
nodes = [node for node in gathered if node is not None]
spider = NodeSpiderCrawl(self.protocol, self.node, nodes,
self.ksize, self.alpha)
return await spider.find()
async def bootstrap_node(self, addr):
result = await self.protocol.ping(addr, self.node.id)
return Node(result[1], addr[0], addr[1]) if result[0] else None
async def get(self, key):
"""
Get a key if the network has it.
Returns:
:class:`None` if not found, the value otherwise.
"""
log.info("Looking up key %s", key)
dkey = digest(key)
# if this node has it, return it
if self.storage.get(dkey) is not None:
return self.storage.get(dkey)
node = Node(dkey)
nearest = self.protocol.router.findNeighbors(node)
if len(nearest) == 0:
log.warning("There are no known neighbors to get key %s", key)
return None
spider = ValueSpiderCrawl(self.protocol, node, nearest,
self.ksize, self.alpha)
return await spider.find()
async def set(self, key, value):
"""
Set the given string key to the given value in the network.
"""
if not check_dht_value_type(value):
raise TypeError(
"Value must be of type int, float, bool, str, or bytes"
)
log.info("setting '%s' = '%s' on network", key, value)
dkey = digest(key)
return await self.set_digest(dkey, value)
async def set_digest(self, dkey, value):
"""
Set the given SHA1 digest key (bytes) to the given value in the
network.
"""
node = Node(dkey)
nearest = self.protocol.router.findNeighbors(node)
if len(nearest) == 0:
log.warning("There are no known neighbors to set key %s",
dkey.hex())
return False
spider = NodeSpiderCrawl(self.protocol, node, nearest,
self.ksize, self.alpha)
nodes = await spider.find()
log.info("setting '%s' on %s", dkey.hex(), list(map(str, nodes)))
# if this node is close too, then store here as well
biggest = max([n.distanceTo(node) for n in nodes])
if self.node.distanceTo(node) < biggest:
self.storage[dkey] = value
ds = [self.protocol.callStore(n, dkey, value) for n in nodes]
# return true only if at least one store call succeeded
return any(await asyncio.gather(*ds))
def saveState(self, fname):
"""
Save the state of this node (the alpha/ksize/id/immediate neighbors)
to a cache file with the given fname.
"""
log.info("Saving state to %s", fname)
data = {
'ksize': self.ksize,
'alpha': self.alpha,
'id': self.node.id,
'neighbors': self.bootstrappableNeighbors()
}
if len(data['neighbors']) == 0:
log.warning("No known neighbors, so not writing to cache.")
return
with open(fname, 'wb') as f:
pickle.dump(data, f)
@classmethod
def loadState(self, fname):
"""
Load the state of this node (the alpha/ksize/id/immediate neighbors)
from a cache file with the given fname.
"""
log.info("Loading state from %s", fname)
with open(fname, 'rb') as f:
data = pickle.load(f)
s = Server(data['ksize'], data['alpha'], data['id'])
if len(data['neighbors']) > 0:
s.bootstrap(data['neighbors'])
return s
def saveStateRegularly(self, fname, frequency=600):
"""
Save the state of node with a given regularity to the given
filename.
Args:
fname: File name to save retularly to
frequency: Frequency in seconds that the state should be saved.
By default, 10 minutes.
"""
self.saveState(fname)
loop = asyncio.get_event_loop()
self.save_state_loop = loop.call_later(frequency,
self.saveStateRegularly,
fname,
frequency)
def check_dht_value_type(value):
"""
Checks to see if the type of the value is a valid type for
placing in the dht.
"""
typeset = set(
[
int,
float,
bool,
str,
bytes,
]
)
return type(value) in typeset

View File

@ -1,115 +0,0 @@
from operator import itemgetter
import heapq
class Node:
def __init__(self, node_id, ip=None, port=None):
self.id = node_id
self.ip = ip
self.port = port
self.long_id = int(node_id.hex(), 16)
def sameHomeAs(self, node):
return self.ip == node.ip and self.port == node.port
def distanceTo(self, node):
"""
Get the distance between this node and another.
"""
return self.long_id ^ node.long_id
def __iter__(self):
"""
Enables use of Node as a tuple - i.e., tuple(node) works.
"""
return iter([self.id, self.ip, self.port])
def __repr__(self):
return repr([self.long_id, self.ip, self.port])
def __str__(self):
return "%s:%s" % (self.ip, str(self.port))
class NodeHeap:
"""
A heap of nodes ordered by distance to a given node.
"""
def __init__(self, node, maxsize):
"""
Constructor.
@param node: The node to measure all distnaces from.
@param maxsize: The maximum size that this heap can grow to.
"""
self.node = node
self.heap = []
self.contacted = set()
self.maxsize = maxsize
def remove(self, peerIDs):
"""
Remove a list of peer ids from this heap. Note that while this
heap retains a constant visible size (based on the iterator), it's
actual size may be quite a bit larger than what's exposed. Therefore,
removal of nodes may not change the visible size as previously added
nodes suddenly become visible.
"""
peerIDs = set(peerIDs)
if len(peerIDs) == 0:
return
nheap = []
for distance, node in self.heap:
if node.id not in peerIDs:
heapq.heappush(nheap, (distance, node))
self.heap = nheap
def getNodeById(self, node_id):
for _, node in self.heap:
if node.id == node_id:
return node
return None
def allBeenContacted(self):
return len(self.getUncontacted()) == 0
def getIDs(self):
return [n.id for n in self]
def markContacted(self, node):
self.contacted.add(node.id)
def popleft(self):
if len(self) > 0:
return heapq.heappop(self.heap)[1]
return None
def push(self, nodes):
"""
Push nodes onto heap.
@param nodes: This can be a single item or a C{list}.
"""
if not isinstance(nodes, list):
nodes = [nodes]
for node in nodes:
if node not in self:
distance = self.node.distanceTo(node)
heapq.heappush(self.heap, (distance, node))
def __len__(self):
return min(len(self.heap), self.maxsize)
def __iter__(self):
nodes = heapq.nsmallest(self.maxsize, self.heap)
return iter(map(itemgetter(1), nodes))
def __contains__(self, node):
for _, n in self.heap:
if node.id == n.id:
return True
return False
def getUncontacted(self):
return [n for n in self if n.id not in self.contacted]

View File

@ -1,128 +0,0 @@
import random
import asyncio
import logging
from rpcudp.protocol import RPCProtocol
from .kademlia.node import Node
from .kademlia.routing import RoutingTable
from .kademlia.utils import digest
log = logging.getLogger(__name__)
class KademliaProtocol(RPCProtocol):
def __init__(self, sourceNode, storage, ksize):
RPCProtocol.__init__(self)
self.router = RoutingTable(self, ksize, sourceNode)
self.storage = storage
self.sourceNode = sourceNode
def getRefreshIDs(self):
"""
Get ids to search for to keep old buckets up to date.
"""
ids = []
for bucket in self.router.getLonelyBuckets():
rid = random.randint(*bucket.range).to_bytes(20, byteorder='big')
ids.append(rid)
return ids
def rpc_stun(self, sender):
return sender
def rpc_ping(self, sender, nodeid):
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
return self.sourceNode.id
def rpc_store(self, sender, nodeid, key, value):
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
log.debug("got a store request from %s, storing '%s'='%s'",
sender, key.hex(), value)
self.storage[key] = value
return True
def rpc_find_node(self, sender, nodeid, key):
log.info("finding neighbors of %i in local table",
int(nodeid.hex(), 16))
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
node = Node(key)
neighbors = self.router.findNeighbors(node, exclude=source)
return list(map(tuple, neighbors))
def rpc_find_value(self, sender, nodeid, key):
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
value = self.storage.get(key, None)
if value is None:
return self.rpc_find_node(sender, nodeid, key)
return {'value': value}
async def callFindNode(self, nodeToAsk, nodeToFind):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.find_node(address, self.sourceNode.id,
nodeToFind.id)
return self.handleCallResponse(result, nodeToAsk)
async def callFindValue(self, nodeToAsk, nodeToFind):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.find_value(address, self.sourceNode.id,
nodeToFind.id)
return self.handleCallResponse(result, nodeToAsk)
async def callPing(self, nodeToAsk):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.ping(address, self.sourceNode.id)
return self.handleCallResponse(result, nodeToAsk)
async def callStore(self, nodeToAsk, key, value):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.store(address, self.sourceNode.id, key, value)
return self.handleCallResponse(result, nodeToAsk)
def welcomeIfNewNode(self, node):
"""
Given a new node, send it all the keys/values it should be storing,
then add it to the routing table.
@param node: A new node that just joined (or that we just found out
about).
Process:
For each key in storage, get k closest nodes. If newnode is closer
than the furtherst in that list, and the node for this server
is closer than the closest in that list, then store the key/value
on the new node (per section 2.5 of the paper)
"""
if not self.router.isNewNode(node):
return
log.info("never seen %s before, adding to router", node)
for key, value in self.storage.items():
keynode = Node(digest(key))
neighbors = self.router.findNeighbors(keynode)
if len(neighbors) > 0:
last = neighbors[-1].distanceTo(keynode)
newNodeClose = node.distanceTo(keynode) < last
first = neighbors[0].distanceTo(keynode)
thisNodeClosest = self.sourceNode.distanceTo(keynode) < first
if len(neighbors) == 0 or (newNodeClose and thisNodeClosest):
asyncio.ensure_future(self.callStore(node, key, value))
self.router.addContact(node)
def handleCallResponse(self, result, node):
"""
If we get a response, add the node to the routing table. If
we get no response, make sure it's removed from the routing table.
"""
if not result[0]:
log.warning("no response from %s, removing from router", node)
self.router.removeContact(node)
return result
log.info("got successful response from %s", node)
self.welcomeIfNewNode(node)
return result

View File

@ -1,185 +0,0 @@
import heapq
import time
import operator
import asyncio
from collections import OrderedDict
from .kademlia.utils import OrderedSet, sharedPrefix, bytesToBitString
class KBucket:
def __init__(self, rangeLower, rangeUpper, ksize):
self.range = (rangeLower, rangeUpper)
self.nodes = OrderedDict()
self.replacementNodes = OrderedSet()
self.touchLastUpdated()
self.ksize = ksize
def touchLastUpdated(self):
self.lastUpdated = time.monotonic()
def getNodes(self):
return list(self.nodes.values())
def split(self):
midpoint = (self.range[0] + self.range[1]) / 2
one = KBucket(self.range[0], midpoint, self.ksize)
two = KBucket(midpoint + 1, self.range[1], self.ksize)
for node in self.nodes.values():
bucket = one if node.long_id <= midpoint else two
bucket.nodes[node.id] = node
return (one, two)
def removeNode(self, node):
if node.id not in self.nodes:
return
# delete node, and see if we can add a replacement
del self.nodes[node.id]
if len(self.replacementNodes) > 0:
newnode = self.replacementNodes.pop()
self.nodes[newnode.id] = newnode
def hasInRange(self, node):
return self.range[0] <= node.long_id <= self.range[1]
def isNewNode(self, node):
return node.id not in self.nodes
def addNode(self, node):
"""
Add a C{Node} to the C{KBucket}. Return True if successful,
False if the bucket is full.
If the bucket is full, keep track of node in a replacement list,
per section 4.1 of the paper.
"""
if node.id in self.nodes:
del self.nodes[node.id]
self.nodes[node.id] = node
elif len(self) < self.ksize:
self.nodes[node.id] = node
else:
self.replacementNodes.push(node)
return False
return True
def depth(self):
vals = self.nodes.values()
sp = sharedPrefix([bytesToBitString(n.id) for n in vals])
return len(sp)
def head(self):
return list(self.nodes.values())[0]
def __getitem__(self, node_id):
return self.nodes.get(node_id, None)
def __len__(self):
return len(self.nodes)
class TableTraverser:
def __init__(self, table, startNode):
index = table.getBucketFor(startNode)
table.buckets[index].touchLastUpdated()
self.currentNodes = table.buckets[index].getNodes()
self.leftBuckets = table.buckets[:index]
self.rightBuckets = table.buckets[(index + 1):]
self.left = True
def __iter__(self):
return self
def __next__(self):
"""
Pop an item from the left subtree, then right, then left, etc.
"""
if len(self.currentNodes) > 0:
return self.currentNodes.pop()
if self.left and len(self.leftBuckets) > 0:
self.currentNodes = self.leftBuckets.pop().getNodes()
self.left = False
return next(self)
if len(self.rightBuckets) > 0:
self.currentNodes = self.rightBuckets.pop(0).getNodes()
self.left = True
return next(self)
raise StopIteration
class RoutingTable:
def __init__(self, protocol, ksize, node):
"""
@param node: The node that represents this server. It won't
be added to the routing table, but will be needed later to
determine which buckets to split or not.
"""
self.node = node
self.protocol = protocol
self.ksize = ksize
self.flush()
def flush(self):
self.buckets = [KBucket(0, 2 ** 160, self.ksize)]
def splitBucket(self, index):
one, two = self.buckets[index].split()
self.buckets[index] = one
self.buckets.insert(index + 1, two)
def getLonelyBuckets(self):
"""
Get all of the buckets that haven't been updated in over
an hour.
"""
hrago = time.monotonic() - 3600
return [b for b in self.buckets if b.lastUpdated < hrago]
def removeContact(self, node):
index = self.getBucketFor(node)
self.buckets[index].removeNode(node)
def isNewNode(self, node):
index = self.getBucketFor(node)
return self.buckets[index].isNewNode(node)
def addContact(self, node):
index = self.getBucketFor(node)
bucket = self.buckets[index]
# this will succeed unless the bucket is full
if bucket.addNode(node):
return
# Per section 4.2 of paper, split if the bucket has the node
# in its range or if the depth is not congruent to 0 mod 5
if bucket.hasInRange(self.node) or bucket.depth() % 5 != 0:
self.splitBucket(index)
self.addContact(node)
else:
asyncio.ensure_future(self.protocol.callPing(bucket.head()))
def getBucketFor(self, node):
"""
Get the index of the bucket that the given node would fall into.
"""
for index, bucket in enumerate(self.buckets):
if node.long_id < bucket.range[1]:
return index
def findNeighbors(self, node, k=None, exclude=None):
k = k or self.ksize
nodes = []
for neighbor in TableTraverser(self, node):
notexcluded = exclude is None or not neighbor.sameHomeAs(exclude)
if neighbor.id != node.id and notexcluded:
heapq.heappush(nodes, (node.distanceTo(neighbor), neighbor))
if len(nodes) == k:
break
return list(map(operator.itemgetter(1), heapq.nsmallest(k, nodes)))

View File

@ -1,97 +0,0 @@
import time
from itertools import takewhile
import operator
from collections import OrderedDict
class IStorage:
"""
Local storage for this node.
IStorage implementations of get must return the same type as put in by set
"""
def __setitem__(self, key, value):
"""
Set a key to the given value.
"""
raise NotImplementedError
def __getitem__(self, key):
"""
Get the given key. If item doesn't exist, raises C{KeyError}
"""
raise NotImplementedError
def get(self, key, default=None):
"""
Get given key. If not found, return default.
"""
raise NotImplementedError
def iteritemsOlderThan(self, secondsOld):
"""
Return the an iterator over (key, value) tuples for items older
than the given secondsOld.
"""
raise NotImplementedError
def __iter__(self):
"""
Get the iterator for this storage, should yield tuple of (key, value)
"""
raise NotImplementedError
class ForgetfulStorage(IStorage):
def __init__(self, ttl=604800):
"""
By default, max age is a week.
"""
self.data = OrderedDict()
self.ttl = ttl
def __setitem__(self, key, value):
if key in self.data:
del self.data[key]
self.data[key] = (time.monotonic(), value)
self.cull()
def cull(self):
for _, _ in self.iteritemsOlderThan(self.ttl):
self.data.popitem(last=False)
def get(self, key, default=None):
self.cull()
if key in self.data:
return self[key]
return default
def __getitem__(self, key):
self.cull()
return self.data[key][1]
def __iter__(self):
self.cull()
return iter(self.data)
def __repr__(self):
self.cull()
return repr(self.data)
def iteritemsOlderThan(self, secondsOld):
minBirthday = time.monotonic() - secondsOld
zipped = self._tripleIterable()
matches = takewhile(lambda r: minBirthday >= r[1], zipped)
return list(map(operator.itemgetter(0, 2), matches))
def _tripleIterable(self):
ikeys = self.data.keys()
ibirthday = map(operator.itemgetter(0), self.data.values())
ivalues = map(operator.itemgetter(1), self.data.values())
return zip(ikeys, ibirthday, ivalues)
def items(self):
self.cull()
ikeys = self.data.keys()
ivalues = map(operator.itemgetter(1), self.data.values())
return zip(ikeys, ivalues)

View File

@ -1,57 +0,0 @@
"""
General catchall for functions that don't make sense as methods.
"""
import hashlib
import operator
import asyncio
async def gather_dict(d):
cors = list(d.values())
results = await asyncio.gather(*cors)
return dict(zip(d.keys(), results))
def digest(s):
if not isinstance(s, bytes):
s = str(s).encode('utf8')
return hashlib.sha1(s).digest()
class OrderedSet(list):
"""
Acts like a list in all ways, except in the behavior of the
:meth:`push` method.
"""
def push(self, thing):
"""
1. If the item exists in the list, it's removed
2. The item is pushed to the end of the list
"""
if thing in self:
self.remove(thing)
self.append(thing)
def sharedPrefix(args):
"""
Find the shared prefix between the strings.
For instance:
sharedPrefix(['blahblah', 'blahwhat'])
returns 'blah'.
"""
i = 0
while i < min(map(len, args)):
if len(set(map(operator.itemgetter(i), args))) != 1:
break
i += 1
return args[0][:i]
def bytesToBitString(bites):
bits = [bin(bite)[2:].rjust(8, '0') for bite in bites]
return "".join(bits)

View File

@ -0,0 +1,5 @@
from libp2p.io.exceptions import IOException
class RawConnError(IOException):
pass

View File

@ -0,0 +1,21 @@
from abc import abstractmethod
from typing import Tuple
import trio
from libp2p.io.abc import Closer
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.stream_muxer.abc import IMuxedConn
class INetConn(Closer):
muxed_conn: IMuxedConn
event_started: trio.Event
@abstractmethod
async def new_stream(self) -> INetStream:
...
@abstractmethod
def get_streams(self) -> Tuple[INetStream, ...]:
...

Some files were not shown because too many files have changed in this diff Show More