Compare commits

...

1281 Commits

Author SHA1 Message Date
Kevin Mai-Husan Chia
d2c2a5f933
Merge pull request #431 from wolfgang/patch-1
Don't hash data passed to nacl library in Ed25519PublicKey.verify
2021-03-17 15:45:46 +08:00
Wolfgang Deutsch
0a9f8d23a3
Update ed25519.py
Don't hash data passed to nacl library in Ed25519PublicKey.verify
2021-03-16 08:55:44 +01:00
Kevin Mai-Husan Chia
2efc072ad9
Merge pull request #425 from MohamedAlFahim/patch-1
Fix broken release notes link
2021-02-25 13:13:59 +08:00
Kevin Mai-Husan Chia
55e6ed6e6c
Merge pull request #426 from McSinyx/trio-lowlevel
Use trio.lowlevel instead of trio.hazmat
2021-02-25 13:11:35 +08:00
Nguyễn Gia Phong
080f8edc8e Use trio.lowlevel instead of trio.hazmat
Since trio 0.15.0, hazmat has been deprecated.

trio-typing and mypy are bumped to support newer trio and each other.
2021-02-23 22:02:34 +07:00
Piper Merriam
12786f4e26
Merge pull request #424 from g-r-a-n-t/master
Interop tests updated and fixed.
2020-09-01 19:51:21 -06:00
Mohamed Al-Fahim
69ffaa08d7
Fix broken release notes link 2020-08-27 01:01:45 +00:00
Grant Wuerker
36a4a9150d
Interop tests updated and fixed. 2020-08-25 15:59:22 -06:00
Alex Stokes
5144ab8289
Merge pull request #411 from libp2p/release-notes-fix
Release notes fix: v0.1.15 -> v0.1.5
2020-03-25 16:35:14 -07:00
Jason Carver
e712995024 Release notes fix: v0.1.15 -> v0.1.5
Also, update to the actual release date.
2020-03-25 15:59:45 -07:00
Jason Carver
612a2f7c51
Bump version: 0.1.4 → 0.1.5 2020-03-25 15:50:57 -07:00
Alex Stokes
c5e1d3263f
add release notes for v0.1.15 2020-03-23 18:06:40 -07:00
Alex Stokes
379a157d6b
Merge pull request #410 from ralexstokes/configurable-msg-id
Allow `Pubsub` creator to supply a custom msg_id
2020-02-28 08:42:49 -08:00
Alex Stokes
857cc29d27
add content-addressed message id utility 2020-02-28 08:25:03 -08:00
Alex Stokes
ef666267bd
Allow Pubsub creator to supply a custom msg_id 2020-02-28 08:25:03 -08:00
Kevin Mai-Husan Chia
9d68de8c21
Merge pull request #406 from mhchia/feature/noise-patterxx
Noise: `PatternXX` and testing with go
2020-02-28 17:13:00 +08:00
mhchia
ec8c10965d
travis CI: use the latest go version 2020-02-28 15:50:03 +08:00
mhchia
1af12ddf43
interop test: support both noise and plaintext 2020-02-28 15:50:03 +08:00
mhchia
cb4e1115c6
Remove wrong encoding 2020-02-28 15:50:03 +08:00
mhchia
c1f6054b3c
plaintext: update pb 2020-02-28 15:50:03 +08:00
mhchia
728d9e5c9c
Noise: update pb 2020-02-28 15:50:02 +08:00
mhchia
f8240bd2cb
plaintext: use varint msg read/writer 2020-02-28 15:50:02 +08:00
mhchia
6016ea731b
BaseMsgReadWriter
- Change `BaseMsgReadWriter` to encode/decode messages with abstract
method, which can be implemented by the subclasses. This allows us to
create subclasses `FixedSizeLenMsgReadWriter` and
`VarIntLenMsgReadWriter`.
2020-02-28 15:50:01 +08:00
mhchia
88f660a9c5
Noise: try to use noise in go 2020-02-28 15:50:01 +08:00
mhchia
a9f4f285ee
Noise: clean up dup code 2020-02-28 15:50:01 +08:00
mhchia
13e8f496a7
Noise: add noise option in the factories and tests 2020-02-28 15:50:01 +08:00
mhchia
1d2a976597
Remove TODOs 2020-02-28 15:50:00 +08:00
mhchia
48a7c1a969
Security: ensure remote pubkeys are stored 2020-02-28 15:50:00 +08:00
mhchia
e02bba93b1
Noise: fix docs
`connection.py` is removed.
2020-02-28 15:49:59 +08:00
mhchia
f0df2d189f
Plaintext: use existing msgio reader 2020-02-28 15:49:59 +08:00
mhchia
3c2e835725
Security: SecureSession
Make security sessions(secio, noise) share the same implementation
`BaseSession` to avoid duplicate implementation of buffered read.
2020-02-28 15:49:59 +08:00
mhchia
2df47a943c
Refactor 2020-02-28 15:49:59 +08:00
mhchia
874c6bbca4
Refactor MsgIOReadWriter
- Abstract it as `MsgReadWriter`
- `MsgIOReadWriter` as a subclass of `MsgReadWriter`
2020-02-28 15:49:58 +08:00
mhchia
ea645f0bd6
Copied read/write from secio 2020-02-28 15:49:58 +08:00
mhchia
f8739268e2
Noise: encrypt and decrypt in NoiseConnection
TODO: Add a buffer to read only `n` bytes in `read(n)`
2020-02-28 15:49:58 +08:00
mhchia
6ea96e9313
Add comments in pattern xx 2020-02-28 15:49:57 +08:00
mhchia
d0290d2b5a
Noise: complete handshake process
TODO
- Figure out why `state.rs` is erased at some moment(even handshake
is not done).
- Refactor
- Add tests
2020-02-28 15:49:56 +08:00
mhchia
8a4ebd4cbb
Support read/write of noise msg and payload 2020-02-28 15:49:56 +08:00
Kevin Mai-Husan Chia
1f881e0464
Merge pull request #405 from mhchia/feature/noise
Noise: skeleton of transport and connection
2020-02-28 15:44:49 +08:00
mhchia
4620544d45
Noise: fix docs 2020-02-15 12:35:36 +08:00
mhchia
f4c545ed68
isort: add noise to 3rd party config 2020-02-15 12:18:44 +08:00
mhchia
95959725db
Noise pattern: Fix flake8 2020-02-15 12:18:19 +08:00
mhchia
d7fabab3e1
Noise: add compiled pb2.py 2020-02-09 13:23:12 +08:00
mhchia
0324a69841
Noise: add PatternXX 2020-02-09 00:33:26 +08:00
mhchia
f27db83a14
Noise: add TODO comments 2020-02-08 10:48:29 +08:00
mhchia
fb53edbc04
Change async def write
To return `None` instead of `int. `Writer.write` *does* write all data
in all use case.
2020-02-08 10:24:37 +08:00
Kevin Mai-Husan Chia
1152f9b703
Update libp2p/security/noise/transport.py
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
2020-02-08 10:14:35 +08:00
mhchia
23ece34157
test_noise_connection: fix missing nursery 2020-02-07 18:17:15 +08:00
Kevin Mai-Husan Chia
99f505d6d7
Merge pull request #403 from hukkinj1/disable-incremental-mypy
Disable incremental mode of mypy
2020-02-07 18:10:34 +08:00
mhchia
897e66b7e1
Add the skeletons of noise transport and conn 2020-02-07 17:47:50 +08:00
Taneli Hukkinen
2f3c5b2ee0 Disable incremental mode of mypy 2020-02-06 13:21:36 +01:00
Kevin Mai-Husan Chia
e63584c387
Merge pull request #404 from libp2p/feature/trio
Merge `feature/trio` into `master`
2020-02-06 10:49:53 +08:00
mhchia
f1dbd52d67
Merge branch 'master' into feature/trio 2020-02-06 10:39:54 +08:00
Kevin Mai-Husan Chia
7c74e36d41
Merge pull request #365 from mhchia/feature/porting-to-trio
Porting to trio
2020-02-06 10:14:11 +08:00
mhchia
ddbedc6c15
Pubsub: handle_talk
- Change from async function to sync
- Change the name to `notify_subscriptions`, which is clearer.
2020-02-05 21:44:33 +08:00
mhchia
5b03a7ad9f
Mplex: only close the send of new stream channel 2020-02-05 21:41:28 +08:00
mhchia
b7c2ec2187
Mplex: change the reference url
To the commit hash, to make it more correct.
2020-02-05 21:31:04 +08:00
mhchia
7f8c0f11f6
Pubsub: change channel size
To `32` to conform to the go implementation.
2020-02-05 21:30:26 +08:00
mhchia
ba0fb8a833
Fix: use pass over ...(Ellipse)
Use `...`(Ellipse) only in abstract methods.
2020-02-05 20:36:42 +08:00
mhchia
1fff6ad6b4
Mplex: change message channel size to 8
To avoid infinity sized channel, and to conform to the go
implementation.
2020-02-05 20:31:18 +08:00
mhchia
64c9c48dac
Mplex: change new stream channel size
To `0`, i.e. no unbuffered, to avoid growing buffer size.
2020-02-05 19:48:02 +08:00
mhchia
1e7d5c73ee
test_mplex_stream: refactor 2020-02-05 17:25:39 +08:00
mhchia
996b5cf15d
Mplex: catch exceptions from channel.send 2020-02-05 17:05:30 +08:00
mhchia
12cb0d9ac4
Swarm: change notify_xxx back to async func 2020-02-04 22:56:13 +08:00
mhchia
13930ae718
SwarmConn: perform close right away
In `_handle_new_streams`, when the underlying muxed conn is unavailable,
close `SwarmConn` itself right away, to reset all the streams.
Therefore, the stream processed by `_handle_muxed_stream` are conscious
of the fact that they are reset. It allows a more graceful clean up.
2020-02-04 22:51:21 +08:00
mhchia
c0ab609559
Mplex: catch RawConnError when writing
Also, do nothing in `MplexStream.reset` if `MuxedConnUnavailable` is
raised when sending the message.
2020-02-04 21:57:59 +08:00
mhchia
f884bfa39e
SwarmConn: don't access Swarm.manager
Open a local nursery instead.
2020-02-04 21:57:11 +08:00
mhchia
0548d28568
Fix: StreamReset in the stream handlers
Since we don't catch `Exception` in the stream handlers, catch them in
the stream handlers in the tests.
2020-02-04 20:46:40 +08:00
mhchia
a7ba59bf9f
Add a nursery in Swarm
To avoid using the one in `Service`
2020-02-04 20:45:58 +08:00
mhchia
b007bb4d07
Use the latest async-service 2020-02-04 17:46:30 +08:00
mhchia
89338914d3
Add comment for serve_tcp 2020-02-04 17:45:56 +08:00
mhchia
857bb34f4e
Add checkpoints in PubsubNotifee
Since some of the methods in `PubsubNotifee` are doing nothing,
add checkpoints to yield control.
2020-02-04 17:45:37 +08:00
mhchia
66975ae3f2
Pubsub: change run_task to run_daemon_task 2020-02-04 17:43:39 +08:00
mhchia
7ae9de9002
Fix handler in net_stream_pair_factory
Change it to async function. It wasn't discovered since we caught all
exceptions raised in stream handlers.
2020-02-04 17:09:26 +08:00
mhchia
3a91f114ab
Swarm: add default_stream_handler
Advantage:
- To avoid `None` checks
- If users forget to register a stream handler for `Swarm`,
with the default stream handler, opened streams aren't removed
until `Swarm` finishes.
2020-02-04 17:05:53 +08:00
mhchia
3fc60cb312
SwarmConn: iterate streams.copy in _cleanup
To avoid `RuntimeError` if `streams` is changed.
2020-02-04 17:04:28 +08:00
mhchia
d483982acb
SwarmConn: don't catch exceptions in handler 2020-02-04 15:10:49 +08:00
mhchia
5da102d1c9
Ping protocol: move with statement out of try 2020-02-04 15:09:42 +08:00
mhchia
113696dce2
TravisCI: use python 3.7 instead of 3.7-dev 2020-02-03 16:04:32 +08:00
mhchia
22963a3099
Fix trio-typing>=0.3,<0.4
To be consistent with trinity
2020-02-02 18:18:01 +08:00
mhchia
05d5d045ea
Fix pubsub interop: missing unsubscribe_fn 2020-02-02 18:17:22 +08:00
mhchia
1588be2be9
Change the channel size of peer queue
Back to `0`, to avoid unlimited buffer size.
2020-01-31 17:42:47 +08:00
mhchia
e57d01f360
PR feedback
- Use f-string
- Fix wrongly indented comments
- Add dep `trio-typing`
2020-01-28 15:48:41 +08:00
mhchia
095a848f30
Add clean-up logics into TrioSubscriptionAPI
Register an `unsubscribe_fn` when initializing the TrioSubscriptionAPI.
`unsubscribe_fn` is called when subscription is unsubscribed.
2020-01-28 00:29:05 +08:00
mhchia
c3ba67ea87
Remove locks in PubsubNotifee
- Change `open_memory_channel(0)` to `open_memory_channel(math.inf)`, to
avoid `peer_queue.send` and `dead_peer_queue.send` blocking. This allows
us to remove the locks.
- Only catch `trio.BrokenResourceError`, which is caused by Pubsub when
it's closing.
2020-01-27 14:30:44 +08:00
mhchia
92ea35e147
Fix IPubsub and add IPubsub.wait_until_ready 2020-01-27 00:10:33 +08:00
mhchia
e3a1dd62e4
Use new type hinting for trio channel 2020-01-26 23:56:19 +08:00
mhchia
42bc4d5d06
INetworkService implement ServiceAPI 2020-01-26 23:55:31 +08:00
mhchia
ddfbf9ffc8
Use raise from to reserve stacktrace 2020-01-26 23:54:29 +08:00
mhchia
b85bab1a09
Don't catch trio.BusyResourceError 2020-01-26 23:09:56 +08:00
mhchia
5b4b65faa8
Change default value of read()
From `n = -1` to `n = None`, to comply with trio API
2020-01-26 23:03:38 +08:00
mhchia
6e01a7da31
PR feedback: async with host.run() 2020-01-26 16:44:42 +08:00
Kevin Mai-Husan Chia
8e0972f256
Merge pull request #389 from ShadowJonathan/f-string_clean
Apply f-string formatting to everything (except logging)
2020-01-20 18:41:20 +08:00
Jonathan de Jong
4c3510f738 Merge branch 'master' into f-string_clean
# Conflicts:
#	libp2p/network/connection/raw_connection.py
2020-01-20 11:25:37 +01:00
Kevin Mai-Husan Chia
0a2d86a3d0
Merge pull request #392 from ShadowJonathan/lru_lint
add lru to known_third_parties (to fix windows lint)
2020-01-20 11:20:12 +08:00
mhchia
dcc4aa52fc
Merge branch 'master' into feature/porting-to-trio 2020-01-19 16:56:40 +08:00
mhchia
f0c4254bbd
Use Service instead of ServiceAPI
To fix error with async-service==0.1.0a5
2020-01-18 00:31:39 +08:00
mhchia
6c7aa30191
Add events in Pubsub
To ensure `handle_peer_queue` and `handle_dead_peer_queue` are indeed
run before the tests finish. Previously, we get errors when performing
`iter_dag` after cancellation. This is because `handle_peer_queue` or
`handle_dead_peer_queue` is not actually run before the Service is
cancelled.
2020-01-18 00:17:30 +08:00
mhchia
54871024cc
Pin the version of async-service to a4 2020-01-16 18:54:19 +08:00
mhchia
eef241e70e
Make Mplex and SwarmConn not Service
After second thoughts, they seem not a good candidate of `Service`.
The shutdown logic becomes simpler by making them not `Service`.
2020-01-07 21:50:03 +08:00
mhchia
eab59482c0
Use the real get_unused_tcp_port
To get rid of the fake one
2020-01-07 16:45:59 +08:00
mhchia
45eeb4fba3
Change notify_xxx to sync functions
Since we already have `Swarm.run_task`, we can just change notify
functions to sync.
2020-01-07 16:45:06 +08:00
mhchia
4db043a26a
Remove pexpect from tox 2020-01-07 16:23:00 +08:00
mhchia
52f85586b8
Fix docs 2020-01-07 15:41:44 +08:00
mhchia
fe4354d377
Fix tests_interop
- Remove pexpect
- Use new version of `p2pclient`, which makes use of anyio
- Clean up tests
2020-01-07 14:14:34 +08:00
mhchia
000e777ac7
Try older async-service 2019-12-26 20:44:58 +08:00
mhchia
94f0fcb6ad
Iterate dead_peer_receive_channel with async for 2019-12-26 20:44:32 +08:00
mhchia
68c84b273d
Use cls over the name of the factory 2019-12-26 20:44:10 +08:00
mhchia
3c98b1973d
Remove useless conftest for pubsub 2019-12-26 20:43:38 +08:00
NIC Lin
4d814f0587
Merge pull request #394 from NIC619/fix_closing_a_reset_connection
Fix `close` an already reset connection
2019-12-26 12:13:26 +08:00
NIC619
3b0386d861
Raise RawConnError 2019-12-24 22:27:16 +08:00
NIC619
c62f1f374f
Check if transport is closing before write/close 2019-12-24 22:23:38 +08:00
NIC619
a390d21385
Remove RawConnError from exception handling 2019-12-24 21:57:13 +08:00
mhchia
fb6076c061
Upgrade to 0.1.0a4
Probably it can solve the dag issue:
https://github.com/ethereum/async-service/issues/12
2019-12-24 21:50:42 +08:00
mhchia
6ae3f5dc1b
Add checkpoints in tests 2019-12-24 21:28:37 +08:00
mhchia
53dbb0aff1
Fix pubsub_notifee.py
For wrong syntax and import
2019-12-24 18:37:59 +08:00
mhchia
573c049d0f
Catch expections in PubsubNotifee
Also, add lock to avoid resource race condition
2019-12-24 18:31:39 +08:00
mhchia
2287dc95be
Fix test for info_from_p2p_addr
It is because I removed some checks in the function. This checks should
be useless thanks to mypy
2019-12-24 18:08:33 +08:00
mhchia
3372c32432
Fix examples and modify new_node
- Fix examples `chat.py` and `echo.py`
    - Use trio directly, instead of `trio-asyncio`
    - Remove redundant code
- Change entry API `new_node` to `new_host_trio`
2019-12-24 18:03:18 +08:00
NIC619
8f52315816
Fix close an already reset connection 2019-12-24 16:19:49 +08:00
mhchia
6fe5871d96
Use async-exit-stack over contextlib
For `AsyncExitStack`
2019-12-24 14:44:28 +08:00
mhchia
ce5663705f
Merge branch 'master' into feature/porting-to-trio 2019-12-24 02:19:43 +08:00
Kevin Mai-Husan Chia
c35cb8318a
Merge pull request #393 from ShadowJonathan/hashable_multiaddr
bump multiaddr to 0.0.9
2019-12-23 17:32:08 +08:00
Jonathan de Jong
45ef63af20 bump multiaddr from setup to 0.0.9 2019-12-23 09:24:41 +01:00
Jonathan de Jong
adae81ca01 fix logging bug 2019-12-23 09:15:56 +01:00
Jonathan de Jong
96f6b80e51 Merge remote-tracking branch 'origin/master' into f-string_clean
# Conflicts:
#	libp2p/network/swarm.py
2019-12-23 09:14:15 +01:00
Kevin Mai-Husan Chia
9d3312ebaf
Merge pull request #380 from ShadowJonathan/fix_dependencies
Fix dependency issues (and fastecdsa windows depencency)
2019-12-23 14:26:40 +08:00
Kevin Mai-Husan Chia
1571bfac07
Merge pull request #386 from ShadowJonathan/issue_384
fixes #384
2019-12-23 14:25:38 +08:00
Jonathan de Jong
a08e749150 add lru to known_third_parties 2019-12-21 12:56:44 +01:00
Jonathan de Jong
17074dded0 add tests to new multiple multiaddr change 2019-12-21 10:35:34 +01:00
Jonathan de Jong
0b97f4d57c remove eth_utils per PR request 2019-12-21 08:59:07 +01:00
Jonathan de Jong
df8be6eb09 revert network/tcp.py and add changes to multiaddr_to_socket_fix 2019-12-19 21:09:11 +01:00
Jonathan de Jong
0827d0d9ef add 2 more instances of formatting violations 2019-12-19 18:09:47 +01:00
NIC Lin
28da206aea
Merge pull request #387 from NIC619/fix_inconsistent_pubsub_peer_record_update
Store peer ids in set instead of list and check if peer id exist before access
2019-12-20 00:34:14 +08:00
Jonathan de Jong
b1248ff315 enforced f-strings everywhere, %s on logging
extended _multiaddr_from_socket to support UDP and IPv6 automatically
changed TCPListener to use _ip4_or_6_from_multiaddr to get host, and not ip4 only

enforced `from error` everywhere with raises
added call braces to exceptions
2019-12-19 17:31:18 +01:00
NIC619
3c75c85d7f
Fix extra white space 2019-12-19 23:07:20 +08:00
NIC Lin
cb80cfc50b
Update libp2p/pubsub/gossipsub.py
Co-Authored-By: Chih Cheng Liang <chihchengliang@gmail.com>
2019-12-19 16:33:56 +08:00
NIC619
74092c1371
Apply PR feedback: update error msg 2019-12-19 16:26:37 +08:00
NIC619
e51d376d5e
Combine peers_gossipsub and peers_floodsub 2019-12-19 14:44:49 +08:00
NIC619
6cd3eb8fae
Apply PR feedback:
change param type and remove check before `discard`
2019-12-19 14:15:51 +08:00
Jonathan de Jong
1124fc8211 add eth_utils
add fastecdsa-any requirements
2019-12-19 01:26:44 +01:00
Jonathan de Jong
6cf1b98a88 mark explicit modulo formatting (to get started with PR draft) 2019-12-19 00:37:09 +01:00
Jonathan de Jong
f54bc9d1af Make linter happy 2019-12-18 19:05:22 +01:00
Jonathan de Jong
81fe4049cf Apply PR review feedback
> add `if not addr` clause back
> use f-strings for exceptions instead of %s
2019-12-18 18:47:03 +01:00
Jonathan de Jong
4e4d91b2e2
Apply PR review suggestion (change "muxed" to "network" in docstrings)
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-12-18 10:54:52 +01:00
NIC619
f3732f9480
Fix tests 2019-12-18 12:37:04 +08:00
Jonathan de Jong
6b75901243 apply PR feedback (remote len == 0 block, remove redundant fixme comment + docstring line)
change wording of final SwarmException to include possible case of no addresses in returned address set

add `from error` in except clause
2019-12-17 20:20:09 +01:00
Alex Stokes
445c0f8e65
Dangling kademlia cleanup 2019-12-17 10:11:17 -08:00
Alex Stokes
1355fbae99
Merge pull request #385 from ShadowJonathan/issue_197
fixes #197
2019-12-17 09:35:36 -08:00
NIC619
19ce8a2140
Fix mypy 2019-12-17 21:56:02 +08:00
NIC619
04b9d688f8
Add newsfragment 2019-12-17 19:09:15 +08:00
Jonathan de Jong
3cbe24caab fixes #384
also adds MultiError to libp2p/exceptions.py

and an additional fixme I have noticed
2019-12-17 12:00:11 +01:00
NIC619
474ed41652
Remove dead peer if floodsub write stream fail 2019-12-17 18:48:25 +08:00
NIC619
009df257bc
Check peer id exist in dict before access 2019-12-17 18:47:58 +08:00
mhchia
47d10e186f
Add SubscriptionAPI
And `TrioSubscriptionAPI`, to make subscription io-agnostic.
2019-12-17 18:17:28 +08:00
Jonathan de Jong
794d2101e9 fixes #197 2019-12-17 11:00:45 +01:00
NIC619
f10e3099cb
Change type of peers in pubsub from list to set 2019-12-17 17:55:13 +08:00
NIC619
b4bd997932
Fix mypy 2019-12-17 17:49:49 +08:00
NIC619
65766ec9ac
Change type of local peers var from list to set 2019-12-17 17:36:15 +08:00
NIC619
f1d58ef8ff
Change type of peers from list to set:
`peers_gossipsub`, `peers_floodsub` and mesh/fanout peers
2019-12-17 17:30:24 +08:00
NIC619
7d6daa8e10
Minor cleanup:
- remove outdated comment
- add new peer at the end
- turn peers to send from list to set
2019-12-17 17:17:03 +08:00
mhchia
fb0519129d
Refine Mplex.close and SwarmConn.close
Ensure `close` cleans up things and cancel the service finally.
2019-12-17 15:50:55 +08:00
Jason Carver
ef31f7f6d6
Bump version: 0.1.3 → 0.1.4 2019-12-12 13:59:52 -08:00
Jason Carver
368ac7ef7f Compile release notes 2019-12-12 13:58:41 -08:00
Jason Carver
23fa86979d Added release notes 2019-12-12 13:53:09 -08:00
Alex Stokes
733b1d08b6
Merge pull request #372 from ralexstokes/add-py36-compatibility
Add py36 compatibility
2019-12-10 17:28:49 -08:00
Alex Stokes
3b9d7c7acd
Apply PR feedback 2019-12-10 17:20:41 -08:00
NIC Lin
ad379221b9
Merge pull request #373 from NIC619/refactor_and_cleanup_gossipsub
Refactor and cleanup gossipsub
2019-12-07 16:28:55 +08:00
NIC Lin
a675da52ee
Update libp2p/pubsub/gossipsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-12-07 15:46:42 +08:00
mhchia
d847e78a83
Add dep async-service 2019-12-07 00:19:10 +08:00
mhchia
837a249552
Fix security module 2019-12-07 00:14:01 +08:00
NIC619
2d3bfc8184
Apply PR feedback:
use defaultdict and init control message
2019-12-06 23:42:31 +08:00
mhchia
1929f307fb
Fix all modules except for security 2019-12-06 17:06:37 +08:00
Chih Cheng Liang
82dcce214a
Remove kademlia module (#377)
* Remove kademlia and routing/kademlia

* cleanup

* Fix routed_host test

* lint mypy

* fix doc

* remove set_up_nodes_by_transport_and_disc_opt and fix typing
2019-12-06 14:14:33 +08:00
NIC619
db0017ddbb
Fix lint after applying suggestion 2019-12-05 17:33:07 +08:00
NIC Lin
b4900d53da
Apply suggestions from code review
Co-Authored-By: Chih Cheng Liang <chihchengliang@gmail.com>
2019-12-05 15:21:09 +08:00
NIC619
67f02c512a
Remove unnecessary check and fix test 2019-12-05 15:10:04 +08:00
NIC619
fae3798ca9
Apply PR feedback: correct the comment in test 2019-12-05 14:40:49 +08:00
NIC619
e6813da5f5
Refactor _get_peers_to_send 2019-12-05 14:35:34 +08:00
NIC619
8e591229fd
Update the sleep time in test_handle_prune 2019-12-03 23:10:56 +08:00
NIC619
c08b2375e1
Fix: should not remove topic if no peers 2019-12-03 23:10:47 +08:00
NIC619
a9abf1e3dd
Fix list deletion and add list remove check 2019-12-03 22:37:49 +08:00
NIC619
bb15c817b1
Fix var access before assignment 2019-12-03 22:14:45 +08:00
NIC619
ea6cd30a16
Add back some comment and TODO. Add comment to tests 2019-12-03 18:45:33 +08:00
NIC619
60bd4694a4
Extend wait time for test to pass 2019-12-03 18:03:45 +08:00
mhchia
e9ab0646e3
Fix Pubsub 2019-12-03 17:27:49 +08:00
NIC619
b405fd76e9
Add test for gossip heartbeat 2019-12-03 15:49:58 +08:00
NIC619
8dec0b111d
Add test for mesh heartbeat 2019-12-03 15:49:45 +08:00
NIC619
5efdf4c703
Group messages for peer in heartbeat 2019-12-03 15:48:23 +08:00
NIC619
ab1500c708
Remove unneccessary check in gossip heartbeat 2019-12-03 15:03:06 +08:00
Alex Stokes
d516cf51b8
Add py3.6 to travis config 2019-12-02 16:33:32 -08:00
Alex Stokes
a1fd106bf3
Ensure correct names of test envs in circle ci 2019-12-02 16:33:32 -08:00
Alex Stokes
4c0f511516
Add py36 tox env for testing 2019-12-02 16:33:32 -08:00
Alex Stokes
63fd531ed0
Fixes to add python 3.6 compatibility 2019-12-02 16:33:32 -08:00
NIC619
920cf646ef
Fix lint and add check in fanout heartbeat 2019-12-02 22:49:27 +08:00
NIC619
a7e0c5d737
Add missing cleanup in gossipsub remove_peer 2019-12-02 22:41:49 +08:00
NIC619
357341e0d8
Remove unneccessary filter and check in gossipsub 2019-12-02 22:40:35 +08:00
NIC619
c2d88962c7
Add gossipsub heartbeat_initial_delay 2019-12-02 16:55:16 +08:00
NIC619
0672f5ae6d
Fix: move heartbeat delay to heartbeat 2019-12-02 16:38:48 +08:00
mhchia
bdbb7b2394
Add RoutedHostFactory
And skip the tests for `RoutedHost` for now, since there are too many to
be fixed in `Kademlia`, and it's not that necessary now.
2019-12-01 19:17:44 +08:00
mhchia
eb494e8682
Fix ping protocol 2019-12-01 19:17:32 +08:00
mhchia
6149aacc01
Fix examples 2019-12-01 17:55:07 +08:00
mhchia
31bf774a16
Fix tests in protocol_muxer and libp2p 2019-12-01 17:43:14 +08:00
mhchia
62e47080f5
Fix tests/network 2019-12-01 16:51:06 +08:00
mhchia
79fcdf3a02
Update tests in test_tcp.py
Besides, run `make format`
2019-12-01 16:26:16 +08:00
NIC619
50fd0acf41
Cleanup outdated TODOs in gossipsub 2019-11-30 20:19:17 +08:00
NIC619
0a52a05375
Del entry if no more peers subscribe to the topic 2019-11-30 20:02:11 +08:00
NIC619
e59ac6a250
Cleanup TODOs in pubsub 2019-11-30 17:12:37 +08:00
NIC Lin
dfdcf524b7
Merge pull request #362 from NIC619/add_signing_and_verification_to_pubsub
Add signing and verification to pubsub
2019-11-30 13:44:12 +08:00
NIC619
658a0ae156
Apply PR feedback:
move signature validation logic into signature validator
2019-11-29 19:37:48 +08:00
mhchia
1e600ea7e0
Fix Mplex and Swarm 2019-11-29 19:09:56 +08:00
NIC619
1c54c38ca7
Fix lint and add signing_strict to interop tests 2019-11-29 17:24:40 +08:00
NIC619
f4e86b1172
Add tests for failed signature validation cases 2019-11-29 14:13:07 +08:00
NIC619
a262b94836
Apply PR feedback:
check if signing key and ID match
2019-11-29 14:12:42 +08:00
NIC619
064c109b64
Fix signature validator:
Add prefix and return verify result
2019-11-28 18:45:00 +08:00
Jason Carver
14bcc2a7a7
Bump version: 0.1.2 → 0.1.3 2019-11-27 16:20:36 -08:00
Jason Carver
3c0675bbba
Fix bumpversion to look for double-quotes 2019-11-27 16:19:24 -08:00
Jason Carver
52ec6a1606 Compile release notes 2019-11-27 16:13:20 -08:00
Jason Carver
f28227729c Customize release notes title
towncrier likes to capitalize things, so libp2p was showing up as
Libp2P. Uuuugly.
2019-11-27 16:09:27 -08:00
Jason Carver
40973ea106 Fix "previous" version in bumpversion to: v0.1.2 2019-11-27 16:04:51 -08:00
Jason Carver
7466ace0b8
Merge pull request #370 from libp2p/remove-unbuildable-dependencies-from-docs-extra
Remove unbuildable dependencies from docs extra
2019-11-27 14:37:10 -08:00
Jason Carver
ce37082c44 Add release note for #318
Note that these commits are really just a bugfix, but from a release
perspective, it's the first time that docs will be publicly available.
2019-11-27 14:17:13 -08:00
Jason Carver
27ecd4b0ed Mock dependencies that are excluded in readthedocs
It seems preferable to import just fastecdsa. But if you do that,
then some kind of side-effect doesn't happen, which means that `sec1` is
not available as an attribute on `fastecdsa.encoding`.

So we specifically mock the sub-modules as well.
2019-11-27 14:07:14 -08:00
Jason Carver
3e5f883c50 Strip out fastecdsa in setup.py, during doc build 2019-11-27 14:07:14 -08:00
Jason Carver
4f95bdce42
Merge pull request #368 from carver/release-notes
Add release notes for v0.1.3
2019-11-27 10:49:02 -08:00
Jason Carver
57f1f49a0f Set the latest version in release notes 2019-11-27 10:38:08 -08:00
Jason Carver
84b548beae Back-generate release notes for v0.1.3 release 2019-11-27 10:38:08 -08:00
Jason Carver
2a9dc96269
Merge pull request #369 from carver/docs-travis-target
Add docs test to Travis CI
2019-11-27 10:36:15 -08:00
NIC619
d5d6962dce
Update Pubsub fixture and test 2019-11-27 17:15:24 +08:00
NIC619
0fd400fdf8
Sign and verify in Pubsub 2019-11-27 17:15:24 +08:00
NIC619
f3c9972159
Implement Pubsub signature validator 2019-11-27 17:15:24 +08:00
NIC619
683710573e
Add strict_signing: bool and sign_key to Pubsub 2019-11-27 17:15:23 +08:00
NIC Lin
da10fc8531
Merge pull request #360 from NIC619/store_our_keypair_in_peerstore
Store our keypair in peerstore
2019-11-27 17:14:36 +08:00
Jason Carver
abf0da925c Built docs for modules 2019-11-26 17:44:33 -08:00
Jason Carver
d503950179 Fix all doc build warnings 2019-11-26 17:44:33 -08:00
Jason Carver
6668e8d339 sphinx to v2, because starting from blank slate 2019-11-26 17:29:13 -08:00
Jason Carver
1564ad659b Convert doc warnings into errors for CI 2019-11-26 17:20:24 -08:00
Jason Carver
b672da82ec Fix version parse failure because of double-quote 2019-11-26 17:19:19 -08:00
Jason Carver
69742da4a3 Add docs test to Travis CI 2019-11-26 16:50:48 -08:00
Jason Carver
a6864e3bd3
Merge pull request #366 from carver/splice-in-template
Splice in project template
2019-11-26 15:48:17 -08:00
Jason Carver
5d3be4f890 Fix comment about mypy being not semver 2019-11-26 15:38:32 -08:00
Jason Carver
1a283d0b1a Fix merge bug in the readme 2019-11-26 15:23:10 -08:00
Jason Carver
f84edfc133 Add missing package test script used by Makefile 2019-11-26 13:33:58 -08:00
Jason Carver
d589daf00d Fill in template variables 2019-11-26 13:33:57 -08:00
Jason Carver
d1b5a56ccf Match linting rules after merging in template 2019-11-26 13:33:08 -08:00
Jason Carver
27da312285 Splice in project template
I tried to merge it so that future updates from the template will be
much easier.

The template is hosted at:
https://github.com/ethereum/ethereum-python-project-template
2019-11-26 13:30:41 -08:00
NIC Lin
cac7e3909a
Merge pull request #357 from NIC619/minor_fix_replace_del
Replace (check and) del pattern with pop method
2019-11-26 19:39:41 +08:00
mhchia
ec43c25b45
Rewrite factories, made some of the test running 2019-11-26 19:24:30 +08:00
NIC619
9f3f2c53da
Apply PR feedback:
use defaultdict for peer data map
2019-11-26 16:18:15 +08:00
Chih Cheng Liang
417b5e7d61
remove unused asyncio 2019-11-26 15:27:06 +08:00
Chih Cheng Liang
50db9e1474
add setup.py 2019-11-26 15:27:06 +08:00
Chih Cheng Liang
6ab0e108d3
minor 2019-11-26 15:27:06 +08:00
Chih Cheng Liang
a397ccdc04
makes test_mplex_stream.py::test_mplex_stream_read_write work 2019-11-26 15:27:06 +08:00
Chih Cheng Liang
c55ea0e5bb
implement trio queue interface 2019-11-26 15:27:05 +08:00
Chih Cheng Liang
41ff884eef
rewrite tcp reader/writer interface 2019-11-26 15:27:05 +08:00
Chih Cheng Liang
d4d345c3c7
progressing 2019-11-26 15:27:05 +08:00
Chih Cheng Liang
f5c725788e
need manual stop 2019-11-26 15:27:05 +08:00
Chih Cheng Liang
ed17bfd663
hack chat example 2019-11-26 15:27:04 +08:00
NIC619
94c7a0bca4
Apply PR feedback and fix 2019-11-26 14:04:28 +08:00
NIC619
fab27b0357
Fix lint 2019-11-26 12:35:50 +08:00
NIC619
ffa73f5649
Check if pubkey matches peer ID before add 2019-11-26 12:33:55 +08:00
NIC619
0bfbdf7fab
Move keypair into PeerData 2019-11-26 12:27:59 +08:00
NIC619
828ae69c66
Apply PR feedback: add_key_pair 2019-11-26 11:52:31 +08:00
Jason Carver
89363b2d9b Add internal type for release notes 2019-11-25 21:44:29 +01:00
Jason Carver
07eecb5c69 Separate release-note build from release 2019-11-25 21:44:29 +01:00
Christoph Burgdorf
02fe35663c Setup towncrier to generate release notes 2019-11-25 21:44:29 +01:00
Alex Stokes
493224b75e
Merge pull request #359 from ralexstokes/update-protobuf-dep
Update `protobuf` dep to `3.10.0`
2019-11-25 09:53:46 -08:00
Alex Stokes
7850018041
Update protobuf dep to >=3.10.0 2019-11-25 09:45:30 -08:00
NIC619
076dae50f3
Fix missing BasicHost interface update 2019-11-25 22:05:33 +08:00
NIC619
566e4c080d
Fix lint 2019-11-25 17:32:53 +08:00
NIC619
e28a974425
Update peer store initialization in host factory 2019-11-25 17:17:09 +08:00
NIC619
8b4022328d
Update peer store interface 2019-11-25 17:16:47 +08:00
NIC619
e49de15227
Add get_private_key to Host 2019-11-25 16:57:00 +08:00
NIC619
a63f00d8f8
Store our pubkey/privkey info during
`initialize_default_swarm`
2019-11-25 16:55:55 +08:00
NIC619
144d93a023
Add pubkey/privkey info to PeerStore 2019-11-24 18:04:00 +08:00
NIC619
9837f30698
Rename peer_map to peer_data_map 2019-11-24 16:52:46 +08:00
NIC619
e355cb2600
Apply PR feedback:
Only use pop method if error handling is in place
2019-11-23 16:04:22 +08:00
NIC619
501eef59de
Apply PR feedback:
Only use pop method if graceful failure handling is desired
2019-11-21 14:48:03 +08:00
Chih Cheng Liang
bcd7890124
Move test utilities to tools (#356)
* move test factories to libp2p/tools

* remove unused inits

* move pubsub test utils to tools

* cleanup test_interop

* fix typing libp2p/tools/utils

* add typing to pubsub utils

* fix factories typing

* fix typing for floodsub_integration_test_settings

* fix rest of the typing

* fix isort
2019-11-21 11:47:54 +08:00
NIC619
19907e18ec
Replace (check and) del pattern with pop method 2019-11-20 23:06:37 +08:00
NIC Lin
74198c70b1
Merge pull request #355 from NIC619/check_before_del
Check if entry exists in dictionary before delete
2019-11-19 16:23:26 +08:00
NIC619
c0522c1bd9
Check if entry exists in dictionary before delete 2019-11-17 21:52:05 +08:00
NIC Lin
64c49f809f
Merge pull request #351 from NIC619/handle_stream_io_error
Handle stream io error
2019-11-17 15:09:45 +08:00
NIC619
ace5ef69a8
Apply PR feedback:
handle pubsub dead peer when stream closed in gossipsub
2019-11-16 17:03:04 +08:00
NIC619
ccc7879422
Add stream.write error handling in gossipsub 2019-11-16 16:57:11 +08:00
NIC619
7d1f3d6000
Remove outdated comment 2019-11-16 16:56:59 +08:00
NIC619
1cc5a6f58b
Bump pycryptodome version to 3.9.2 2019-11-16 16:56:59 +08:00
NIC619
cbe57cd5d7
Fix lint 2019-11-16 16:56:59 +08:00
NIC619
86e0fa4563
Handle StreamClosed in ping protocol handler 2019-11-16 16:56:59 +08:00
NIC619
9be9b4bbfc
Handle StreamClosed in pub/gossip/flood-sub 2019-11-16 16:56:59 +08:00
NIC619
c4f9ce6bb3
Handle StreamClosed in identify protocol handler 2019-11-16 16:56:58 +08:00
Jason Carver
32aa20ec39
Merge pull request #353 from carver/release-tools
Pre-release testing tools, plus setup.py updates
2019-11-15 12:05:35 -08:00
Jason Carver
b8ec43a859 remove type:ignore for working pycryptodome type
I didn't dig deeper, it looks like pycryptodome must have fixed their
pkcs1_15.new andd pkcs1_15.verify type signatures.
2019-11-15 11:25:03 -08:00
Jason Carver
cb4b4b8209 setup.py: fix warnings and add description
There were warnings about needing a URL and maintainer.
Also, use the readme as the pypi description.
2019-11-15 11:25:03 -08:00
Jason Carver
444929d5fc Add tool to make package for smoke test 2019-11-15 11:25:03 -08:00
mhchia
56d3e50267
Bump version: 0.1.1 → 0.1.2 2019-11-14 15:29:50 +08:00
NIC Lin
a5c3b8dec2
Merge pull request #340 from NIC619/fix_pubsub_stream_to_disconnected_peer
Register for disconnected event notification by pubsub
2019-11-14 15:16:32 +08:00
NIC619
b8c7f0cfff
Fix lint 2019-11-09 23:55:35 +08:00
NIC619
5dfa29a0df
Track tasks created in pubsub and add close() 2019-11-09 23:24:09 +08:00
NIC619
93ef36bd86
Clean up peer record if pubsub stream fail 2019-11-09 23:24:09 +08:00
NIC619
eeb87848af
Apply PR feedback:
- fix await stream close/reset
- make `_handle_dead_peer` a sync function
2019-11-09 23:24:09 +08:00
NIC619
d36e323703
Update error handling of pubsub stream handler 2019-11-09 23:24:08 +08:00
NIC619
97b3aca535
Fix:
Force context switch before canceling swarm connection tasks
2019-11-09 23:24:08 +08:00
NIC619
a8d9536b08
Spin up handle_dead_peer_queue task 2019-11-09 23:24:08 +08:00
NIC619
84f5210220
Implement handle_dead_peer_queue 2019-11-09 23:24:08 +08:00
NIC619
4b15cb1af5
Implement PubsubNotifee.disconnected 2019-11-09 23:24:08 +08:00
NIC619
c6c9393f2b
Add dead_peer_queue to pubsub 2019-11-09 23:24:08 +08:00
NIC619
3a0c7d06d1
Update comment for connection.close() 2019-11-09 23:24:07 +08:00
Alex Stokes
285bb2ed19
Merge pull request #346 from ralexstokes/add-tests-for-identify
Add tests for identify
2019-11-09 01:51:34 +08:00
Alex Stokes
29873584dc
update test for identify and ping using new test fixture via PR feedback 2019-11-07 21:11:35 -08:00
Alex Stokes
30dee28ef2
add asynccontextmanager utility for a pair of connected hosts 2019-11-07 20:59:01 -08:00
Alex Stokes
2a7b43d853
bugfix: return empty bytes immediately if read length is 0 2019-11-07 20:57:55 -08:00
Alex Stokes
4b01c33d54
add some additional logging 2019-11-07 20:57:43 -08:00
Alex Stokes
9c05e2b0bb
add basic test for identify 2019-11-07 08:05:39 -08:00
Alex Stokes
58f360167d
refactor creation of identify response to make testing easier 2019-11-07 08:05:39 -08:00
Alex Stokes
071eccc995
Merge pull request #345 from ralexstokes/install-default-protocols
Install default protocols
2019-11-08 00:05:05 +08:00
Alex Stokes
9a23609b48
type protocol IDs to satisfy mypy 2019-11-07 07:51:44 -08:00
Alex Stokes
e61a5a677f
clean up tests w/ default protocols in place 2019-11-07 07:51:44 -08:00
Alex Stokes
ed81562a89
Add identify and ping as default protocols 2019-11-07 07:51:43 -08:00
Alex Stokes
cce33b2f50
Merge pull request #337 from ralexstokes/ground-work-for-identify-to-host
Ground work for identify to host
2019-11-07 23:50:36 +08:00
mhchia
10dd997805
Reorganize factories 2019-11-06 12:11:09 -08:00
Alex Stokes
9500bdbf55
Add class attribute for additional property 2019-11-06 12:11:09 -08:00
Alex Stokes
a07c3b0fb0
modify factories to fix tests 2019-11-06 12:11:01 -08:00
Alex Stokes
3deccac2da
fix tests 2019-11-06 12:10:58 -08:00
Alex Stokes
0ca3e83540
lintroll 2019-11-06 11:41:28 -08:00
Alex Stokes
693a8cf99a
Default protocols are constructed using a reference to the host 2019-11-06 11:41:28 -08:00
Alex Stokes
e0a94b6092
identify handler reads data on request from the host 2019-11-06 11:41:27 -08:00
Alex Stokes
32c55bcaf2
hosts track their public key 2019-11-06 11:41:27 -08:00
Alex Stokes
11db313b17
A mux can provide the protocols it responds to 2019-11-06 11:41:27 -08:00
Alex Stokes
07c09d8d9e
Merge pull request #341 from hukkinj1/master
Add more linting
2019-11-07 04:39:08 +09:00
Alex Stokes
d09046f5ca
Merge pull request #343 from ralexstokes/add-pytest-xdist
Add pytest xdist and fix some issues w/ parallelizing tests
2019-11-07 04:34:14 +09:00
Alex Stokes
fbdd52cfdc
Go with immutable datatype in lieu of mutable datatype 2019-11-06 11:25:12 -08:00
Alex Stokes
4a80519ba1
Merge pull request #334 from ralexstokes/add-identify-protocol
Adds identify protocol
2019-11-06 11:05:56 +09:00
Alex Stokes
c30d9ce397
Fix linter error from merge in #315. 2019-11-05 17:50:43 -08:00
Alex Stokes
c8a5f6f8d7
Run docformatter over changes 2019-11-05 17:50:11 -08:00
Alex Stokes
17ba328e1a
Add pytest-xdist plugin to parallelize tests 2019-11-05 17:50:11 -08:00
Alex Stokes
94984be4df
Avoid hard-coding ports where it is not relevant for Kademlia tests 2019-11-05 17:50:11 -08:00
Alex Stokes
5cb4479534
Modify the `KademliaServer` so that lack of port lets the OS choose a free one 2019-11-05 17:50:10 -08:00
Alex Stokes
8ab69b9676
fix medley of typos 2019-11-05 17:37:43 -08:00
Alex Stokes
01dc49164f
fix bug found via typechecker 2019-11-05 17:37:43 -08:00
Alex Stokes
01ebfa760f
Add identify protobufs to makefile 2019-11-05 17:37:43 -08:00
Alex Stokes
08b57fa2ee
Add core handler for identify protocol 2019-11-05 17:37:42 -08:00
Alex Stokes
700209c50a
Merge pull request #315 from ralexstokes/add-ping-protocol
Add `ping` protocol.
2019-11-06 10:14:39 +09:00
Taneli Hukkinen
23848039fd Use pytest.fail() to fail test 2019-11-05 01:26:08 +01:00
Taneli Hukkinen
69279108bc Add flake8-bugbear 2019-11-04 21:16:09 +01:00
Taneli Hukkinen
cab0e0c1c4 Add warn_unreachable=True mypy config 2019-11-04 21:05:12 +01:00
Piper Merriam
dcfeb3e38a
Merge pull request #339 from dmuhs/fix/kademlia-docstring
Fix docstring list in KademliaProtocol
2019-10-29 11:31:35 -06:00
Dominik Muhs
817325f341 Fix docstring list in KademliaProtocol 2019-10-29 18:22:13 +01:00
Piper Merriam
ded3792924
Merge pull request #331 from dmuhs/fix/docs-format
Add automatic docstring formatting
2019-10-28 09:39:52 -06:00
Dominik Muhs
3439a2c10e Add docformatter to lintroll and tox CI check 2019-10-26 12:45:26 +02:00
Alex Stokes
f2bfc68f6d
Re-generate protobufs 2019-10-25 18:51:57 +09:00
Alex Stokes
eaa800c356
Merge pull request #332 from dmuhs/refactor/bool-params
Refactor initiator -> is_initiator and other flags/functions
2019-10-25 09:38:52 +08:00
Dominik Muhs
bafdd8512d Enforce pre-summary newline in docstrings 2019-10-24 20:10:45 +02:00
Dominik Muhs
87ed98d7af Add newline before docstring summary 2019-10-24 20:10:36 +02:00
Dominik Muhs
09ab6c51ec Fix formatting in secio transport 2019-10-24 19:52:03 +02:00
Dominik Muhs
714126faf5 Update secio tests to new initiator flag 2019-10-24 19:30:57 +02:00
Dominik Muhs
b5eeceecbf Rename raw connection initiator flags 2019-10-24 19:28:42 +02:00
Dominik Muhs
5810174374 Rename stream muxer initiator flags 2019-10-24 19:28:19 +02:00
Dominik Muhs
c0452c961b Update stream muxer conftest 2019-10-24 19:25:51 +02:00
Dominik Muhs
031b98ddf0 Rename stream muxer initiator flags 2019-10-24 19:25:34 +02:00
Dominik Muhs
4bda366bb8 Rename protocol muxer initiator flags 2019-10-24 19:25:09 +02:00
Dominik Muhs
021c92ea25 Rename secio initiator flags 2019-10-24 19:22:24 +02:00
Dominik Muhs
7ad96d167c Rename InsecureSession initiator flag 2019-10-24 19:18:29 +02:00
Dominik Muhs
18783b82a2 Rename SecurityMultistream initiator flag 2019-10-24 19:17:00 +02:00
Dominik Muhs
ee0b4daf1c Rename BaseSession initiator flag 2019-10-24 19:16:08 +02:00
Dominik Muhs
afb79da9f8 Fix typo 2019-10-24 19:12:15 +02:00
Dominik Muhs
2e38d5e5fb Rename TransportUpgrader initiator flag 2019-10-24 19:06:49 +02:00
Alex Stokes
477c08da96
Merge pull request #314 from ralexstokes/add-default-protocols
Add default protocols
2019-10-24 20:10:53 +09:00
Alex Stokes
d795a9b854
Add tests for ping protocol 2019-10-24 19:53:04 +09:00
Alex Stokes
1cf239cce6
Respect a remote close during the ping protocol 2019-10-24 19:44:52 +09:00
Alex Stokes
e157c3f654
typing fixes 2019-10-24 18:10:56 +09:00
Alex Stokes
f24b488f79
handle other side closing their end of the connection during ping 2019-10-24 18:10:56 +09:00
Alex Stokes
4a800e5c56
Add first-pass at ping protocol 2019-10-24 18:10:55 +09:00
Alex Stokes
1bf0c31d1f
add basic test for default protocols 2019-10-24 17:36:34 +09:00
Alex Stokes
15cabb1c33
Copy default data to avoid undesirable mutations 2019-10-24 17:29:33 +09:00
Alex Stokes
d0c8b7d8af
Avoid sharing default dictionary argument across all instances of class 2019-10-24 17:29:16 +09:00
Alex Stokes
c92bade815
Add "default protocols" that all hosts have by default 2019-10-24 17:29:16 +09:00
Dominik Muhs
2af97240c5 Fix import formatting 2019-10-24 09:16:58 +02:00
Dominik Muhs
61cd5e5659 Show diff on incorrectly sorted imports 2019-10-24 09:16:49 +02:00
Kevin Mai-Husan Chia
73251a0c36
Merge pull request #328 from NIC619/another_error_handling
Some minor error handlings
2019-10-24 14:53:19 +08:00
Dominik Muhs
3db297fbee Fix mypy errors 2019-10-24 08:53:19 +02:00
Dominik Muhs
eef505f2d9 Add automatic docstring formatter and apply 2019-10-24 08:41:10 +02:00
NIC619
d52b093286
Fix mypy error 2019-10-18 15:59:35 +08:00
NIC619
9889cb8ab1
Fix wrong import 2019-10-18 15:44:07 +08:00
NIC619
211e2f6dd0
Catch PeerDataError in PeerStore.get 2019-10-17 15:52:57 +08:00
NIC619
5063f0e2a6
Fix lint 2019-10-17 15:34:11 +08:00
NIC619
49bd460e37
Catch SedesException in deserialize_public_key 2019-10-17 15:19:39 +08:00
NIC619
da08d37c38
Add SedesException SecioException 2019-10-17 15:03:18 +08:00
NIC619
29bf623d2c
Raise DecryptionFailedException
when failed to decrypt read msg
2019-10-17 14:30:30 +08:00
NIC619
5e5c96f1ea
Add DecryptionFailedException MsgioException 2019-10-17 14:29:58 +08:00
Kevin Mai-Husan Chia
30aeb35122
Merge pull request #324 from aratz-lasa/issue-280
Implemented Host that includes a routing system.
2019-10-16 17:20:23 +09:00
Aratz M. Lasa
ac9feef26c Pull request feedback 2019-10-15 20:32:25 +02:00
Aratz M. Lasa
fa1637850e Removed pipenv Pipfiles 2019-10-15 19:27:04 +02:00
Aratz M. Lasa
70ea471491 Blacked 2019-10-15 19:11:11 +02:00
Aratz M. Lasa
d1d91e4091 Refactored for 'lint' testenv 2019-10-15 19:02:03 +02:00
aratz-lasa
65b5e7aeea
Update libp2p/peer/peerinfo.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-10-15 16:31:20 +02:00
Aratz M. Lasa
8143563831 Added tests for 'RoutedHost' and modified 'FindPeer' 2019-10-15 01:01:16 +02:00
Aratz M. Lasa
3f24b015ab Implemented Host that includes a routing system. Explicitly separating different Host types as in Go implementation 2019-10-14 00:29:28 +02:00
Kevin Mai-Husan Chia
00f83a3694
Merge pull request #321 from mhchia/fix/version-in-setuppy
Fix: version released mismatches setup.py
2019-10-11 22:46:02 +09:00
mhchia
b8963e4433
Fix: version released mismatches setup.py 2019-10-02 16:16:52 +08:00
Kevin Mai-Husan Chia
a369a9f605
Merge pull request #323 from mhchia/fix/ignore-typing-for-wait
Ignore typing for `asyncio.wait`
2019-10-02 16:16:01 +08:00
mhchia
82dc5d9e31
Ignore typing for asyncio.wait 2019-10-02 15:45:54 +08:00
Kevin Mai-Husan Chia
50a0783b98
Merge pull request #320 from aleph-im/master
Remove forced debug level
2019-10-02 11:36:08 +08:00
Moshe Malawach
986a852e7e Remove forced debug level 2019-10-01 11:17:05 +02:00
Alex Stokes
ec015b5a00
Merge pull request #311 from ralexstokes/clean-up-mss-client
Misc cleanups
2019-09-25 11:47:41 -04:00
Alex Stokes
ada4d48b6e
remove overly verbose comments 2019-09-24 15:36:50 -07:00
Alex Stokes
e6a0361e09
Merge pull request #291 from ralexstokes/add-ed25519-support
Adds support for verifying ed25519 signatures, for secio
2019-09-24 10:21:01 -07:00
Alex Stokes
75ec2facce
linter fix 2019-09-24 10:07:33 -07:00
Alex Stokes
673ce40133
Add basic tests for ed25519 keys 2019-09-24 10:06:35 -07:00
Alex Stokes
487c923791
add ed25519 private key deserializer 2019-09-24 10:06:35 -07:00
Alex Stokes
6f638b7afe
Merge pull request #307 from swedneck/patch-1
Add links to the new bridged chats to README.md
2019-09-24 09:54:47 -07:00
Alex Stokes
bbd8279811
Add explicit exception if we are missing a deserializer 2019-09-24 09:51:32 -07:00
Alex Stokes
6e53849604
Delete utils in favor of serialization module 2019-09-24 09:51:31 -07:00
Alex Stokes
5fdca2ffb2
Add public key implementation 2019-09-24 09:51:31 -07:00
Alex Stokes
fa7d1d66a8
Fix import path 2019-09-24 09:51:31 -07:00
Alex Stokes
b142964d31
Adds support for verifying ed25519 signatures, for secio 2019-09-24 09:51:31 -07:00
Kevin Mai-Husan Chia
dbc35e8b9d
Merge pull request #310 from mhchia/fix/missing_await_for_stream_reset_in_pubsub
Add the missing `await` for `stream.reset` in pubsub
2019-09-24 14:04:25 +08:00
Kevin Mai-Husan Chia
b53ca5708f
Merge pull request #305 from mhchia/fix/change-notifee-and-add-tests-for-swarm-conn-and-mplex
Change `Notifee`, add tests for `SwarmConn` and `Mplex`
2019-09-24 14:02:58 +08:00
mhchia
da34b086d5
Merge branch 'master' into fix/change-notifee-and-add-tests-for-swarm-conn-and-mplex 2019-09-24 13:50:54 +08:00
Kevin Mai-Husan Chia
3e9df896f4
Merge pull request #308 from mhchia/fix/move-deps-on-p2pclient-to-tox
Move interop tests outside `tests`
2019-09-24 13:49:35 +08:00
mhchia
b0c919aab7
Add the missing await 2019-09-24 13:34:26 +08:00
mhchia
e217acf4ac
Add additional sleep
To be more confident that the stream handler is registered in the
daemon.
2019-09-24 13:28:25 +08:00
mhchia
7405f078e6
Raise read_delim exception with different msgs
Separate `len(msg_bytes) == 0` and `msg_bytes[-1:] != b"\n"`, to raise
`ParseError` with different messages.
2019-09-24 13:22:25 +08:00
mhchia
37bee9fb16
PR feedback
- Use `TMuxerOptions` and `TSecurityOptions` in libp2p/__init__.py
- Remove the default value for `muxer_transports_by_protocol` in
`MuxerMultistream` and `secure_transports_by_protocol`
`SecureMultistream`
2019-09-24 12:51:59 +08:00
Kevin Mai-Husan Chia
1bd18c84f2
Apply suggestions from code review
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
2019-09-24 12:33:14 +08:00
mhchia
1bfc6b41e4
Add pexpect in isort third party 2019-09-24 12:18:31 +08:00
mhchia
d1c25b8b1e
Fix interop pubsub tests and PR feedback
- Use `from_id`, the changed field name in `PSMessage`.
- PR feedbacks
        - Add label `test` in `testenv` in tox.ini, to avoid wrong
dispatching an environment's command in the future.
        - Use `pytest` over `py.test`.
2019-09-24 11:30:52 +08:00
mhchia
19c17dd512
Remove the leftover test in testenv commands
To make `tox -e py37-interop` run.
2019-09-23 22:10:47 +08:00
mhchia
006002f687
Move interop tests out of tests
It is moved to the top level package `tests_interop`, to avoid circular
dependency, with the dependency moved to `tox`.
2019-09-23 22:00:40 +08:00
mhchia
95ae718e3d
Raise ParseError in read_delim 2019-09-23 16:01:22 +08:00
mhchia
92deae41dc
Change SwarmConn.conn to muxed_conn 2019-09-23 15:46:50 +08:00
mhchia
8d2415a404
Move calls to Notifee inside Swarm 2019-09-23 15:01:58 +08:00
mhchia
6f8394e4bd
Merge branch 'master' into fix/change-notifee-and-add-tests-for-swarm-conn-and-mplex 2019-09-21 18:34:12 +08:00
Kevin Mai-Husan Chia
4a838033ff
Merge pull request #304 from mhchia/fix/detection-of-close
Detect closed `Mplex`
2019-09-21 18:28:16 +08:00
mhchia
539047be2d
Make mplex.read_message handle RawConnError 2019-09-21 18:17:00 +08:00
mhchia
e44c2145cc
Merge branch 'master' into fix/detection-of-close 2019-09-21 18:05:54 +08:00
Kevin Mai-Husan Chia
99ef3f08d8
Merge pull request #303 from mhchia/fix/refactor-mplex-swarm-host
Refactor: `Mplex`, `Swarm`, and `BasicHost`
2019-09-21 17:27:09 +08:00
mhchia
0df578d219
Add the missing exceptions.py 2019-09-21 17:18:55 +08:00
mhchia
a27a817d50
Fix tests 2019-09-20 16:17:13 +08:00
mhchia
89c127eff4
Merge branch 'master' into fix/refactor-mplex-swarm-host 2019-09-20 15:45:28 +08:00
swedneck
c2ef724136
change matrix alias 2019-09-20 06:43:49 +02:00
swedneck
96559cb95a
fix typo 2019-09-20 06:40:50 +02:00
swedneck
d94a36c42b
Add links to the new bridged chats to README.md
matrix, discord, and IRC are all bridged together via matrix
2019-09-19 19:27:18 +02:00
NIC Lin
85457fa308
Merge pull request #299 from NIC619/add_more_error_handling
Add more error handling
2019-09-19 23:45:02 +08:00
NIC619
7fc958e7be
Add exception raised to docstring 2019-09-19 22:19:36 +08:00
NIC619
c6294ad19b
Raise MultiselectCommunicatorError:
when failed to write to communicator
2019-09-19 21:51:23 +08:00
NIC619
b9d1875027
Catch OpenConnectionError in swarm.dial_peer 2019-09-19 21:24:01 +08:00
NIC619
1f76f6ee1b
Raise OpenConnectionError
when failed to open connection
2019-09-19 21:23:35 +08:00
NIC619
c44be5e247
Add OpenConnectionError for base transport 2019-09-19 21:23:01 +08:00
mhchia
62b0bc4580
Remove useless protocol_ids in logging 2019-09-19 16:31:42 +08:00
mhchia
313ae45b45
Add tests for MplexStream 2019-09-19 16:31:42 +08:00
mhchia
02c55e5d14
Add tests for MplexConn 2019-09-19 16:31:42 +08:00
mhchia
a9ad37bc6f
Add mplex tests and fix error in SwarmConn.close 2019-09-19 16:31:41 +08:00
mhchia
d61327f5f9
Add tests for SwarmConn 2019-09-19 16:31:41 +08:00
mhchia
b8b5ac5e06
Add test for notifee disconnected 2019-09-19 16:31:41 +08:00
mhchia
675c61ce3b
Move test_notify from libp2p to network 2019-09-19 16:31:41 +08:00
mhchia
5307c0506b
Change IMuxedConn to INetConn in Notifee 2019-09-19 16:31:41 +08:00
mhchia
0356380996
Add tests for swarm, and debug
Fix `swarm_pair_factory`
2019-09-19 16:31:40 +08:00
mhchia
276ac4d8ab
Add initial test for Swarm.close_peer 2019-09-19 16:31:13 +08:00
mhchia
e7304538da
Add test for Swarm.close_peer 2019-09-19 16:31:12 +08:00
mhchia
6923f257f6
Remove print 2019-09-19 16:07:53 +08:00
mhchia
5f064dd329
Refactor: get rid of single huge _handle_incoming 2019-09-19 15:55:27 +08:00
mhchia
4a689c7d57
Fix error when reset
If `Mplex` is cleanup first, `MplexStream.reset` possibly fails because
`Mplex.streams` is set to `None` in `cleanup`.
2019-09-19 15:55:27 +08:00
mhchia
b51c2939a8
Handle exceptions inside read_message
And remove the need of checking `None` for every read messages.
2019-09-19 15:55:26 +08:00
mhchia
f62f07bb9f
Handle IncompleteRead in handle_incoming 2019-09-19 15:55:26 +08:00
mhchia
2d8e02b7eb
Add detection for disconnections in mplex 2019-09-19 15:55:26 +08:00
mhchia
393b51a744
isort 2019-09-19 15:53:40 +08:00
mhchia
5653b3f604
Add "closed" and "shutting_down" events 2019-09-19 15:53:39 +08:00
mhchia
7cf0495f37
Remove print 2019-09-19 15:38:38 +08:00
mhchia
a7bc9fc358
Asynchronously handling the accepted stream. 2019-09-19 13:59:51 +08:00
mhchia
0c7afeebaf
Fix test_security_multistream 2019-09-19 13:59:50 +08:00
mhchia
6cb033fd1f
Refactor multiselect out of Swarm to BasicHost 2019-09-19 13:59:50 +08:00
mhchia
0bd213bbb7
Refactor mplex and start to add close detection 2019-09-19 13:56:05 +08:00
NIC619
7243eb9766
Fix different exception raised in test 2019-09-17 21:44:48 +08:00
NIC619
f253152858
Handle protocol negotiation failure in swarm new_stream 2019-09-17 16:17:41 +08:00
NIC619
559f419b4e
Fix stream registration in accept_stream 2019-09-17 15:42:18 +08:00
NIC619
0080466d86
Handle RawConnError in InsecureSession.run_handshake 2019-09-16 19:16:41 +08:00
NIC619
09bfa0ab09
Handle IOException in create_secure_session 2019-09-16 19:15:40 +08:00
NIC619
c7593bff97
Add InconsistentNonce in secio 2019-09-16 19:11:46 +08:00
NIC619
359bcf45ff
SecioException inherit from HandshakeFailure 2019-09-16 19:11:06 +08:00
NIC619
d6dda91482
Move HandshakeFailure to libp2p.security 2019-09-16 19:09:09 +08:00
NIC619
4cd5b77f10
Raise RawConnError in RawConnection 2019-09-16 18:37:00 +08:00
NIC619
cb632fa509
Add RawConnError 2019-09-16 18:35:48 +08:00
NIC619
76af835af8
Handle MultiselectError in stream_muxer.accept_stream 2019-09-15 17:35:01 +08:00
NIC619
eaa74c4e26
Handle MultiselectCommunicatorError 2019-09-15 16:58:22 +08:00
NIC619
879f193aa1
Handle errors from
- `read_delim`
    - `read_varint_prefixed_bytes`
    - `decode_uvarint_from_stream`
2019-09-15 16:58:08 +08:00
NIC619
905a473ac3
Add MultiselectCommunicatorError 2019-09-15 16:37:37 +08:00
NIC619
68573e94d3
Have StreamError inherit from IOException 2019-09-15 16:34:16 +08:00
NIC Lin
7483da762e
Merge pull request #298 from NIC619/add_more_logging
Add a little more loggings
2019-09-15 15:34:28 +08:00
NIC619
f368f5e93b
Apply PR feedback 2019-09-15 15:09:58 +08:00
NIC619
501513b747
Update IMultiselectClient 2019-09-14 22:24:53 +08:00
NIC619
9bad7a61f0
Add some loggings to pubsub 2019-09-14 21:54:26 +08:00
NIC619
786a03544c
Add some loggings to swarm and cosmetic updates 2019-09-14 21:47:49 +08:00
NIC619
38f4223e62
Add pip-wheel-metadata to gitignore 2019-09-14 21:47:15 +08:00
NIC619
65a48d5c51
Remove unused select_protocol_or_fail 2019-09-14 17:42:18 +08:00
Kevin Mai-Husan Chia
811cd7813a
Merge pull request #292 from mhchia/fix/dial-sig-changed
Fix `transport.dial` in swarm
2019-09-11 18:56:36 +08:00
mhchia
8c8c206c33
flake8 2019-09-11 18:33:52 +08:00
mhchia
451f993058
Fix isort 2019-09-11 18:05:41 +08:00
mhchia
c1ffc0ab07
Fix transport.dial in swarm 2019-09-11 17:13:21 +08:00
Alex Stokes
cb1a25f94c
Execute the todo to remove an unused argument 2019-09-10 21:01:03 -04:00
Alex Stokes
4bd32cc4bc
Add logs during connection handshake 2019-09-10 19:02:29 -04:00
Alex Stokes
f128c746f0
Write data payload as hex to log 2019-09-10 16:17:40 -04:00
Alex Stokes
68e75707e4
Enhance logs 2019-09-10 16:04:18 -04:00
Kevin Mai-Husan Chia
988ef8c712
Merge pull request #287 from mhchia/fix/mplex-stream-close-reset
Fix close/reset behavior
2019-09-10 23:57:44 +08:00
mhchia
31fb4e0b69
Rewrite _wait_for_data, to handle task precisely
Make the futures first, and then we can compare them with the return
value from `asyncio.wait`.
2019-09-10 23:38:45 +08:00
mhchia
df87f5adb9
Add tests against the daemon for close/reset 2019-09-10 18:01:16 +08:00
mhchia
bb0da41eda
Remove cleanup
`cleanup` cancels all tasks in the loop, including the main one run by
`run_until_complete`
2019-09-10 18:01:16 +08:00
mhchia
a45eb76421
Suppress all exceptions in clean up. 2019-09-10 18:01:16 +08:00
mhchia
e5eb01d22b
Fix stream read 2019-09-10 18:01:16 +08:00
mhchia
df312f3e57
Fix linting 2019-09-10 18:01:15 +08:00
mhchia
0ab548aee5
Add the missing tests 2019-09-10 18:01:15 +08:00
mhchia
be2c0f122a
Fix close behavior 2019-09-10 18:01:14 +08:00
Brian Cloutier
b2146c5268
Don't crash on large messages 2019-09-09 17:13:26 -04:00
Alex Stokes
155bec0562
Fix initiator flag during secio upgrade 2019-09-09 16:40:14 -04:00
Alex Stokes
f9321924ad
Merge pull request #286 from ralexstokes/fix-initiator-flag
Restore `initiator` flag to `BaseSession` type
2019-09-08 16:03:34 -04:00
Alex Stokes
2025a5c7f1
Restore initiator flag to BaseSession type 2019-09-08 15:40:02 -04:00
Alex Stokes
f38899e26e
Merge pull request #284 from ralexstokes/remove-friendly-ids
remove friendly IDs
2019-09-07 11:14:05 -04:00
Alex Stokes
50ae439d20
remove friendly IDs 2019-09-07 11:04:20 -04:00
Alex Stokes
2d1d3d0136
Merge pull request #282 from mhchia/fix/mplex-stream-close-reset
Fix mplex stream close reset
2019-09-06 19:49:03 -04:00
mhchia
6c1f77dc1a
Fix: Change the event.close to event.set
And add missing parts.
2019-09-06 21:35:15 +08:00
mhchia
1cd969a2d5
Fix: Add typing in functions 2019-09-06 20:02:35 +08:00
mhchia
a754e7dbbe
Add the missing tests.constants 2019-09-06 17:59:39 +08:00
mhchia
649a230776
Fix MplexStream.read 2019-09-06 17:26:40 +08:00
mhchia
95926b7376
Temp for mplex_stream 2019-09-06 01:08:42 +08:00
mhchia
207fa75d8f
Add reset and close 2019-09-05 23:44:22 +08:00
mhchia
10415cb956
Use ReadWriteCloser for conns and streams 2019-09-05 23:24:17 +08:00
mhchia
eac159c527
Restructure mplex and mplex_stream 2019-09-05 22:29:33 +08:00
mhchia
96230758e4
Add events in MplexStream
And modify a little bit of `close` and `reset`
2019-09-05 18:18:39 +08:00
Alex Stokes
34b02588cf
Merge pull request #279 from ralexstokes/echo-demo
Echo demo
2019-09-04 19:54:28 +02:00
Alex Stokes
c1fdf9accf
Add echo demo 2019-09-04 10:45:41 -07:00
Alex Stokes
7ed9eac3b6
Merge pull request #278 from ralexstokes/add-buffering-to-secio-reads
Add buffering to secio reads
2019-09-04 19:38:29 +02:00
Alex Stokes
451ec2664a
Address incorrect typing in pycryptodome dependency 2019-09-04 10:19:27 -07:00
Alex Stokes
a099b9c65d
Clean up temporary hold over from previous PR 2019-09-04 10:15:30 -07:00
Alex Stokes
b214f88f75
Avoid using message-based IO in the plaintext protocol
Can reuse the machinery in `secio` but need to generalize the
"buffering" there
2019-09-04 10:12:43 -07:00
Alex Stokes
f86ba7283d
Implement signing for RSA
- mainly for use in `secio` w/ RSA-based identities b/t peers
2019-09-04 10:12:43 -07:00
Alex Stokes
25f504ad35
Allow RSA public key type when deserializing keys 2019-09-04 10:12:43 -07:00
Alex Stokes
1a359770dd
Use msgio IO and proper buffering in secio implementation 2019-09-04 10:12:43 -07:00
Alex Stokes
fc3e3a4be5
Changes to type hints to match new abstractions 2019-09-04 10:12:43 -07:00
Alex Stokes
6d97702da7
Merge pull request #277 from ralexstokes/add-io-abstractions
Introduces IO abstractions and classes for `msgio` IO
2019-09-04 19:11:50 +02:00
Alex Stokes
a764fd4e6f
simplify some of the msgio class hierarchy via PR feedback 2019-09-04 09:57:04 -07:00
Kevin Mai-Husan Chia
e9ca372fb5
Merge pull request #271 from mhchia/fix/pubsub-interop
Pubsub interop with go-libp2p-daemon
2019-09-04 22:28:14 +08:00
mhchia
1f3c9af45b
Add the missing is_proc_running=True 2019-09-04 22:19:11 +08:00
mhchia
34b489af25
Fix kad_peerinfo according to peerinfo 2019-09-04 21:37:33 +08:00
mhchia
0e3d4508d6
PR feedback
- Use `Sequence` instead of `List`
- Add note
- Remove redundant words in docstring
2019-09-04 20:52:18 +08:00
mhchia
db0da8083a
Do p2pd.close if not all of them succeed 2019-09-04 20:52:17 +08:00
Kevin Mai-Husan Chia
51d547ccc5
Update tests/interop/utils.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:38:38 +08:00
Kevin Mai-Husan Chia
b72c489f4e
Update tests/interop/daemon.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:36:42 +08:00
Kevin Mai-Husan Chia
a843514afb
Update tests/interop/daemon.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:35:42 +08:00
Kevin Mai-Husan Chia
155f523c9f
Update tests/interop/daemon.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:33:50 +08:00
Kevin Mai-Husan Chia
bd21b2f66f
Update tests/interop/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:33:29 +08:00
Kevin Mai-Husan Chia
5113785543
Update libp2p/pubsub/pubsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-09-04 20:32:43 +08:00
mhchia
46c13ee1c0
Fix CI to call the script in the correct path 2019-09-04 16:34:56 +08:00
mhchia
dddaacad62
Move install script under tests/interop/go_pkgs 2019-09-04 16:33:25 +08:00
mhchia
677531db76
Fix pubsub tests 2019-09-04 15:33:07 +08:00
Alex Stokes
2a02f92f77
Temporary porcelain until next PR 2019-09-03 21:59:50 -07:00
Alex Stokes
8e8318aa5c
Introduces IO abstractions apart from asyncio or those attached to IRawConnection
Also adds `msgio` utilities to mirror the Go implementation
2019-09-03 21:59:50 -07:00
Alex Stokes
cd3e093001
remove leftover simple security module 2019-09-03 21:38:33 -07:00
Alex Stokes
506093216c
Merge pull request #276 from ralexstokes/fix-secio-impl
Fix secio impl
2019-09-03 23:01:45 +02:00
Alex Stokes
272ab60d47
Merge pull request #275 from ralexstokes/use-different-ecc-backend
Use a different ECC backend `fastecdsa` with a compatible serializer
2019-09-03 23:00:33 +02:00
Alex Stokes
30456f8018
Use a different ECC backend with a compatible serializer
This library has the ``SEC1`` encoder which is compatible
with the serialization of ECC keys/points used in the Go libp2p impl
2019-09-03 13:44:25 -07:00
Alex Stokes
bfd674e22c
Try all pairs of choices, not just a small subset via zip 2019-09-03 13:28:31 -07:00
Alex Stokes
f8bbaf60a1
Add more information to the peer mismatch exception 2019-09-03 13:28:18 -07:00
Alex Stokes
350cc04c97
Allow the key pair to drive ID generation 2019-09-03 13:26:58 -07:00
Alex Stokes
eaeb36c1d9
Merge pull request #274 from ralexstokes/add-identity-hash-for-peer-ids
Add option to inline "short" public keys for peer IDs
2019-09-03 22:21:58 +02:00
Alex Stokes
345e696a7d
Add option to inline "short" public keys for peer IDs
Added to remain interoperable w/ the Go implementation
2019-09-03 13:14:04 -07:00
mhchia
961e51fa2e
Remove leftover prints 2019-09-03 23:39:29 +08:00
mhchia
7f20ab781d
Fix gosssipsub tests 2019-09-03 23:37:34 +08:00
mhchia
d7bce941d8
Fix wrong spelling 2019-09-03 23:36:31 +08:00
mhchia
b23bf5d704
Avoid isort sorting the import wrong 2019-09-03 23:00:31 +08:00
mhchia
4f7bb30d82
Add INetStream to type StreamReader
TODO: Make stream readers implement `Reader`
2019-09-03 22:59:44 +08:00
mhchia
c6d81d70b3
Try 3.7-dev in CI
Use https over ssh when cloning go-libp2p-daemon

To avoid """
Warning: Permanently added the RSA host key for IP address '140.82.113.4' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
"""

Exit if clone fails

Fix daemon url
2019-09-03 22:50:14 +08:00
mhchia
d1b0340164
Update bindings version 2019-09-03 21:55:07 +08:00
mhchia
2ba7948f95
Update bindings version 2019-09-03 18:52:55 +08:00
mhchia
5280f3965c
Update install script for interop
And adjust the structure of go packages for interop
2019-09-03 17:41:17 +08:00
mhchia
749ff275ed
Refactor make_p2pd
Let `make_p2pd` get rid of `unused_tcp_port_factory`, which should only
exist in fixtures/tests.
2019-09-03 16:55:42 +08:00
mhchia
7385a7a677
Add is_gossipsub fixture in interop test
To use the same code to test against both routers: floodsub and
gossipsub.
2019-09-03 16:49:00 +08:00
mhchia
33dae87c35
Add pubsub test for gossipsub 2019-09-03 16:07:44 +08:00
mhchia
fd1f466002
Fix: failed to open stream using existing conn
Fix #233
2019-09-03 14:12:16 +08:00
mhchia
194b494057
Tested against subscriptions and publish 2019-09-02 23:21:57 +08:00
mhchia
3717dc9adf
Add helper functions 2019-09-02 21:01:13 +08:00
mhchia
a883816881
Add connect utility function 2019-09-02 18:40:12 +08:00
mhchia
56ef0b962c
Add test for host connect and disconnect 2019-09-02 17:32:15 +08:00
mhchia
dfd9ebdc5e
Change PeerInfo to remove dep on PeerData 2019-09-02 14:30:27 +08:00
mhchia
b77834d129
Use asyncio.subprocess over pexpect
In the test for pubsub, since there were unknown issues when I test
against pexpect.
2019-09-02 14:30:27 +08:00
mhchia
1b5d064a8d
Add utility functions for libp2p bindings
To prepare for pubsub interop test
2019-09-02 14:30:25 +08:00
Kevin Mai-Husan Chia
db858e467c
Merge pull request #273 from mhchia/fix/secio-pb-initpy
Add the missing __init__.py
2019-09-02 14:27:59 +08:00
mhchia
77a0cc3a87
Add the missing __init__.py 2019-09-02 14:18:52 +08:00
Kevin Mai-Husan Chia
73ae5a35ec
Merge pull request #270 from mhchia/fix/mplex-stream-id-and-interop
Fix mplex stream id and add interop CI
2019-08-31 23:56:43 +08:00
mhchia
b955c0fa02
Explicitly import ID, Transport from secio 2019-08-31 22:38:46 +08:00
mhchia
aa0866698f
PR feedback: Add check in Swarm.close_peer 2019-08-31 22:37:59 +08:00
Kevin Mai-Husan Chia
9e8a6bdf29
Update tests/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-31 22:32:43 +08:00
Kevin Mai-Husan Chia
1e59438f25
Update libp2p/network/swarm.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-31 22:32:32 +08:00
mhchia
cec2aea928
Move shared fixtures and constants to files 2019-08-29 22:38:08 +08:00
mhchia
9ceb5f55bb
Call make_echo_proc with is_host_insecure
Use the fixture, this way we can configure `is_host_insecure` to support
the test against secio.
2019-08-29 22:08:27 +08:00
mhchia
b2c5371323
Add TODO for Swarm.connections 2019-08-29 22:00:07 +08:00
mhchia
c61a06706a
Refactor interop tests and factories
- Add `close` and `disconnect` in `Host`
- Add `close` and `close_peer` in `Network`
- Change `IListener.close` to async, to await for server's closing
- Add factories for security transports, and modify `HostFactory`
2019-08-29 21:38:06 +08:00
mhchia
64c0dab3af
Fix isort 2019-08-29 00:01:48 +08:00
mhchia
e0399beed8
Fix tar argument 2019-08-28 23:50:46 +08:00
mhchia
15f62dff68
Use go12 in CI 2019-08-28 23:48:25 +08:00
mhchia
b726d7c9da
Add tox and CI for interop 2019-08-28 23:39:33 +08:00
mhchia
34a4d7b0ed
Add the missing StreamID class 2019-08-28 21:45:18 +08:00
mhchia
d35b8ffc64
Conform stream_id to go-mplex 2019-08-28 21:43:34 +08:00
Alex Stokes
9b60e1757d
Merge pull request #269 from ralexstokes/add-reset-to-network-stream
Expose `reset` method on `NetStream`
2019-08-26 21:27:57 +02:00
Alex Stokes
a193ae81fd
Fix typo w/ header tag under different reset scenarios 2019-08-26 11:39:30 -07:00
Alex Stokes
c5eda4065d
Expose reset method on NetStream 2019-08-26 11:38:39 -07:00
Alex Stokes
98a0e76dda
Merge pull request #268 from mhchia/fix/mplex-interop
Fix: name of a `MplexStream` is not handled in `Mplex`
2019-08-26 19:51:57 +02:00
Alex Stokes
66c3bacf7e
Merge pull request #254 from ralexstokes/implement-secio
Implement `secio`
2019-08-26 19:22:00 +02:00
Alex Stokes
fa0acd9fc5
Apply PR feedback 2019-08-26 10:03:12 -07:00
Alex Stokes
c1ffb03f77
Update comment to reflect correct function 2019-08-26 09:51:49 -07:00
mhchia
b6c8ab0dc9
Fix #259: Use the unsigned LEB128 impl from py-wasm
Reference: https://github.com/ethereum/py-wasm/blob/master/wasm/parsers/leb128.py
2019-08-26 20:41:10 +08:00
mhchia
d59870ebbf
Fix MplexStream error
When receiving a `NewStream`, the message of that packet is the
name of the stream, which should be handled, rather than letting it go
into the message queue.
2019-08-26 20:39:08 +08:00
NIC Lin
5b122d04b2
Merge pull request #267 from NIC619/fix_conn_attr_in_mplex
Small fix on `conn` attribute and docstring in mplex
2019-08-25 16:53:29 +08:00
NIC619
5e68aff1d1
Fix conn attribute and docstring in mplex 2019-08-25 14:42:44 +08:00
Alex Stokes
f08aa339b4
Verify the channel can pass some plaintext 2019-08-24 23:26:26 +02:00
Alex Stokes
737195f461
Simplify testing connection w/ other simplifications 2019-08-24 23:15:31 +02:00
Alex Stokes
10e30beb42
Disable "friendly" IDs for tests that expect a full string 2019-08-24 22:57:22 +02:00
Alex Stokes
a363ba97d1
Work in terms of the IRawConnection abstraction 2019-08-24 22:52:09 +02:00
Alex Stokes
eb5ef39399
Convert message IO to work w/ a RawConnection. 2019-08-24 22:47:56 +02:00
Alex Stokes
7c004a4e14
Mypy fixes 2019-08-24 22:38:26 +02:00
Alex Stokes
44e5de636f
Add "friendly" peer ID string representation for debugging 2019-08-24 22:38:26 +02:00
Alex Stokes
852609c85d
Clean up base session type 2019-08-24 22:38:26 +02:00
Alex Stokes
9355f33da8
Add basic test for secio
Two peers in-memory can create a secure, bidirectional channel
2019-08-24 22:38:26 +02:00
Alex Stokes
b8c0ef9ebb
Fix bugs in secio implementation 2019-08-24 22:38:25 +02:00
Alex Stokes
228032805a
Some code cleanup 2019-08-24 22:38:25 +02:00
Alex Stokes
3f4589d497
Get tests working 2019-08-24 22:38:25 +02:00
Alex Stokes
d176115972
Add secio to security upgrader suite 2019-08-24 22:37:46 +02:00
Alex Stokes
376a5d4fc6
Adjust callsite 2019-08-24 22:37:45 +02:00
Alex Stokes
1adef05e94
Typing and linting fixes 2019-08-24 22:37:45 +02:00
Alex Stokes
0fa3331b8c
Add clearer indication of "self encryption" 2019-08-24 22:37:45 +02:00
Alex Stokes
8e913a3faa
Dispatch serialization of keys based on key type
- Add some tests to check high-level roundtrip
2019-08-24 22:37:45 +02:00
Alex Stokes
4d30b31c55
Finish first pass at secio implementation 2019-08-24 22:37:45 +02:00
Alex Stokes
af2e50aaf4
Add facility for authenticated encryption 2019-08-24 22:37:45 +02:00
Alex Stokes
228c17ae9e
Add ECDH key exchange utility 2019-08-24 22:37:44 +02:00
Alex Stokes
fb13dfa7b3
Add sign and verify operations for secp256k1 keys 2019-08-24 22:37:44 +02:00
Alex Stokes
3c97a5a0ed
Add ECC key implementation 2019-08-24 22:37:44 +02:00
Alex Stokes
91e11f3ec0
[wip] more secio implementation 2019-08-24 22:37:44 +02:00
Alex Stokes
0cc3fc24a7
Add source for some secure bytes, e.g. to provide a nonce 2019-08-24 22:37:44 +02:00
Alex Stokes
b59c5d6ca1
Add "msgio" functions 2019-08-24 22:37:44 +02:00
Alex Stokes
26165b0729
[wip] sketch of secio handshake 2019-08-24 22:37:44 +02:00
Alex Stokes
fd08bcf624
Add secio protobufs 2019-08-24 22:37:43 +02:00
Alex Stokes
23f53ef954
Allow optional peer ID in a security session 2019-08-24 22:37:43 +02:00
Alex Stokes
27c0a4f77b
formatting 2019-08-24 22:37:43 +02:00
Alex Stokes
3e07faf343
Merge pull request #266 from ralexstokes/remove-unrelated-cde
Remove test suite for `asyncio`, *not* `py-libp2p`
2019-08-24 22:22:06 +02:00
Alex Stokes
d764ca2884
Merge pull request #265 from ralexstokes/add-wait-closed
Add call to `wait_closed` method of asyncio.StreamWriter
2019-08-24 22:17:29 +02:00
Alex Stokes
7ccdeaf308
Remove test suite for asyncio, *not* py-libp2p 2019-08-24 22:10:47 +02:00
Alex Stokes
5b32bfdd3f
Add call to wait_closed method of asyncio.StreamWriter 2019-08-24 22:06:24 +02:00
Alex Stokes
d9883ee4f0
Merge pull request #263 from ralexstokes/remove-stream-from-connection
Encapsulate the concept of a stream to the stream multiplexer
2019-08-24 21:58:59 +02:00
Alex Stokes
9c5fb4fa5a
Encapsulate concept of a "stream id" to a "muxed" connection 2019-08-24 21:50:07 +02:00
Alex Stokes
e29c1507bf
remove unused fields 2019-08-24 21:50:06 +02:00
Alex Stokes
9a74797068
Merge pull request #264 from ralexstokes/remove-simple-security
Removes the SimpleSecurityTransport
2019-08-24 21:49:29 +02:00
Alex Stokes
73495038e1
remove simple security 2019-08-24 21:39:25 +02:00
Alex Stokes
1790e48c99
Remove deprecated file 2019-08-24 21:36:30 +02:00
Kevin Mai-Husan Chia
da3c8be464
Merge pull request #253 from mhchia/feature/plaintext-2.0.0
Add `/plaintext/2.0.0` secure channel
2019-08-22 23:40:02 +08:00
mhchia
b516579256
Remove the unnecessary RSAPrivateKey.from_bytes 2019-08-22 22:54:14 +08:00
mhchia
c1eacf221f
PR feedback
- Check if the received peer id matches the one we initialize the
session with.
- Move the check inside `run_handshake`
2019-08-22 22:53:49 +08:00
Kevin Mai-Husan Chia
7c630df610
Update libp2p/security/insecure/transport.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-22 22:53:49 +08:00
mhchia
16a4fd33c1
PR feedbacks
- Move exceptions to exceptions.py
- Raise `UpgradeFailure` in upgrader
- Refine the try/catch for upgraders in swarm
2019-08-22 22:53:49 +08:00
mhchia
3e04480d62
Raise HandshakeFailure in transport
Change the exception handling flow.
Raise `SecurityUpgradeFailure` in security_multistream.
2019-08-22 22:53:48 +08:00
mhchia
80452d9589
Fix make_exchange_message to use the new API 2019-08-22 22:53:48 +08:00
mhchia
921bfb65cc
Verify the remote pubkey and peer_id
- Add `from_bytes` in RSAPublicKey and Secp256k1PublicKey
- Add `pubkey_from_protobuf` to parse pubkey from protobuf
- Verify key and peer_id in `InsecureSession.run_handshake`
2019-08-22 22:53:48 +08:00
mhchia
ef476e555b
Use RawConnection.read
Instead of accessing its reader and writer directly.

TODO: considering add `ReaderWriterCloser` interface and let connection
and stream inherit from it.
2019-08-22 22:53:48 +08:00
mhchia
0b466ddc86
Add lock to RawConnection
To avoid `self.writer.drain()` is called in parallel.
Reference: https://bugs.python.org/issue29930
2019-08-22 22:53:47 +08:00
mhchia
5768daa9bf
PR feedbacks
- Nits
- Add `SecurityUpgradeFailure` and handle `UpgradeFailure` in Swarm.
2019-08-22 22:53:47 +08:00
Kevin Mai-Husan Chia
2a1367b011
Apply suggestions from code review
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-22 22:53:47 +08:00
mhchia
0b62321265
Fix test_security_multistream
By passing initiator keypairs to node.
2019-08-22 22:53:47 +08:00
mhchia
de8d356955
Fix tests failure due to lack of peer id
Fix it through doing plaintext handshake.
2019-08-22 22:53:47 +08:00
mhchia
bb7d37fd4f
Fix msg encoding
- Change varint-prefix encode to fixedint-prefix(4 bytes) encode.
2019-08-22 22:53:46 +08:00
mhchia
22b1a5395d
A working plaintext 2.0 without validation 2019-08-22 22:53:46 +08:00
mhchia
a0923d202a
Move varint and delim read/write to toplevel
To `libp2p.utils`.
2019-08-22 22:53:46 +08:00
mhchia
5192944724
Update pb 2019-08-22 22:53:46 +08:00
mhchia
59b373b48a
Add plaintext.proto
Update Makefile to handle the import in `plaintext.proto`.
Import path is modified to be relative to the project root.
And we run `protoc` from where `Makefile` locates, i.e. the project
root.

Reference:
- plaintext.proto: 62b2c6c482/sec/insecure/pb/plaintext.proto
2019-08-22 22:53:46 +08:00
mhchia
7bc363f2fa
Remove initiator in Mplex
Besides, fix the wrong passed `multi_addr` to `mplex_stream`.
2019-08-22 22:53:45 +08:00
Kevin Mai-Husan Chia
8217319c28
Merge pull request #258 from NIC619/adjust_coincurve_version_to_match_trinity
Adjust coincurve version to match Trinity's
2019-08-22 22:19:25 +08:00
NIC619
87f5a86950
Adjust coincurve version 2019-08-22 18:01:41 +08:00
Kevin Mai-Husan Chia
46b4a446c3
Merge pull request #257 from mhchia/fix/add-missing-initpy
Fix the issue when importing from other package
2019-08-22 16:32:34 +08:00
mhchia
2752e7a04b
Ignore mypy error here 2019-08-22 15:58:38 +08:00
mhchia
dbd4ed44dd
Fix the issue when importing from other package 2019-08-22 15:36:17 +08:00
Kevin Mai-Husan Chia
6b05d9ab8f
Merge pull request #252 from mhchia/fix/add-negotiation-when-upgrading-to-mplex
Negotiate multiplexer protocol when upgrading to `MuxedConn`
2019-08-21 12:30:22 +08:00
mhchia
550c23f9f9
PR feedback
- Use the order in `MuxerMultistream` as the precedence in multiselect
2019-08-21 11:43:25 +08:00
mhchia
8596f7390f
PR feedback: set protocol_id to constants 2019-08-21 11:43:24 +08:00
mhchia
d7d8440b2c
PR feedback: nitpicks 2019-08-21 11:43:24 +08:00
mhchia
4358a4bc89
Negotiate multiselect version for Muxer
`MuxerMultistream` is introduced to negotiate `Multiselect` version
before negotiating Multiplexer's version. This is required by
multistream 1.x
2019-08-21 11:43:24 +08:00
Alex Stokes
20aed4430e
Merge pull request #256 from ralexstokes/patch-up-key-to-bytes
Internalize the protobuf serialization to the concept of a `Key`
2019-08-20 10:33:33 -07:00
Alex Stokes
e1d3f1601f
Satisfy mypy 2019-08-20 19:28:32 +02:00
Alex Stokes
87d943aa39
Internalize the protobuf serialization to the concept of a Key
Given its use across various components of `libp2p` (not just peer IDs),
it makes the abstraction cleaner to pull the serialization into the
key class and expose the canonical serialization to bytes.
2019-08-20 19:01:36 +02:00
mhchia
5d611801c7 Fix isort 2019-08-18 19:51:04 +08:00
mhchia
86d4ce1da8 Add delim_encode and delim_read
- Add `StreamCommunicator` and `RawConnectionCommunicator`, read/write
messages with delim codec, with `IMuxedStream` and `IRawConnection`
respectively.
- Use it in `Multiselect` and `MultiselectClient`.
2019-08-18 19:51:04 +08:00
mhchia
8cd23abfe2 Remove the leftover merge related stuff 2019-08-16 11:03:16 +08:00
mhchia
e293b89925 Fix mypy
It seems the stub doesn't allow default value for params
2019-08-16 11:03:16 +08:00
mhchia
c5f32bf431 PR feedback for MplexStream.read 2019-08-16 11:03:16 +08:00
Kevin Mai-Husan Chia
8699568d43 Update libp2p/stream_muxer/mplex/mplex_stream.py
Co-Authored-By: Alex Stokes <r.alex.stokes@gmail.com>
2019-08-16 11:03:16 +08:00
mhchia
f2c31f6fe3 Fix isort 2019-08-16 11:03:16 +08:00
mhchia
b27cd0f24f Use bytearray over bytes
To avoid copies.
2019-08-16 11:03:16 +08:00
mhchia
92320523d5 Add the missing exceptions.py 2019-08-16 11:03:16 +08:00
mhchia
e37b8bcf19 mypy: Add read_buffer_nonblocking in Mplex 2019-08-16 11:03:16 +08:00
mhchia
f281e3e1db flake8 2019-08-16 11:03:16 +08:00
mhchia
9f8276fa84 Support read(n=-1)
Now, `n=-1` indicates that we want to read until EOF. However, now we
only read until we have no new message.
2019-08-16 11:03:16 +08:00
mhchia
9cb6ec1c48 Modify the behavior of MplexStream.read 2019-08-16 11:03:16 +08:00
mhchia
2485a00e24 Modify NetStream to read n bytes 2019-08-16 11:03:16 +08:00
Alex Stokes
dbdbcf7440
Merge pull request #250 from ralexstokes/add-secio-groundwork
Add secio groundwork
2019-08-15 19:32:17 -07:00
Alex Stokes
7535a02da7
Clean up key gen 2019-08-15 19:24:30 -07:00
Alex Stokes
d17e6f3392
Fix some test imports that got botched in rebase 2019-08-15 16:46:23 -07:00
Alex Stokes
9a4e23a803
mypy protobuf plugin requires keyword-based initializers 2019-08-15 16:36:32 -07:00
Alex Stokes
82bae341a7
Run isort over files that were missing it 2019-08-15 16:33:35 -07:00
Alex Stokes
2e3ffb9d53
Use types for {Private,Public}Key and address other missing type hints 2019-08-15 16:33:34 -07:00
Alex Stokes
e7d2681fc0
Move base implementations into BaseSession 2019-08-15 16:33:34 -07:00
Alex Stokes
cda74dd382
Update tests for new logic 2019-08-15 16:33:34 -07:00
Alex Stokes
9e18d7561d
Supply local priv and pub key when upgrading to a secure transport 2019-08-15 16:33:34 -07:00
Alex Stokes
7942b7eaa7
Expose writer 2019-08-15 16:33:34 -07:00
Alex Stokes
20dd7d777a
More efficiently remove trailing newline from message 2019-08-15 16:33:34 -07:00
Alex Stokes
0ebc8ffb21
Wire some missing properties up 2019-08-15 16:33:34 -07:00
Alex Stokes
02e073d85a
Keep the host's private key for use in transports 2019-08-15 16:33:33 -07:00
Alex Stokes
879cbf1abd
Add an "insecure session" that satisfies the ISecureConn interface 2019-08-15 16:33:33 -07:00
Alex Stokes
fb43728661
Mark some slow tests as such 2019-08-15 16:33:33 -07:00
Alex Stokes
1e5357a1e1
Update the ISecureConn interface following the reference and simplify accordingly 2019-08-15 16:33:33 -07:00
Alex Stokes
ab7653526f
Code cleanup / formatting 2019-08-15 16:33:33 -07:00
Alex Stokes
b98025c379
Move security transports into their respective sub-packages 2019-08-15 16:33:33 -07:00
Alex Stokes
d50e1b6872
Use direct types over indirect types 2019-08-15 16:33:32 -07:00
Alex Stokes
ff5eaf2429
Merge pull request #249 from ralexstokes/fix-mypy-protos
Add `mypy` protobufs plugin and regenerate protobufs
2019-08-15 16:15:26 -07:00
Alex Stokes
b0e57c2be4
Merge pull request #245 from mhchia/fix/add-initpy-pb
Add the missing __init__.py in pb/
2019-08-15 16:04:09 -07:00
Alex Stokes
0f81ca42a6
Add mypy protobufs plugin and regenerate protobufs 2019-08-15 16:01:44 -07:00
mhchia
97308c897e
Add the missing __init__.py in pb/
Error: https://circleci.com/gh/mhchia/trinity/5342?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
2019-08-15 21:03:45 +08:00
Alex Stokes
125c5d8e2c
Adjust inheritance so that the MRO is clear for the chain of keys 2019-08-14 09:30:23 -07:00
Alex Stokes
3b19104284
Add missing ABC declaration following PR feedback in #240. 2019-08-14 09:25:54 -07:00
Alex Stokes
9977933fd1
Merge pull request #240 from ralexstokes/allow-multiple-identity-types
Allow multiple peer identity types (via different cryptosystems)
2019-08-14 09:13:02 -07:00
Alex Stokes
08e6f2a30c
Update libp2p/crypto/secp256k1.py
pass the secret on to `coincurve` lib

Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-14 09:02:06 -07:00
Alex Stokes
53e583a068
Update libp2p/crypto/secp256k1.py
Pass the secret on to `coincurve` lib

Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-08-14 09:02:06 -07:00
Alex Stokes
67744bcb0f
Add a basic interop test for Go and Python peer IDs 2019-08-14 09:00:40 -07:00
Alex Stokes
ad20d8cb00
Make a KeyPair dataclass for passing around key pairs 2019-08-14 09:00:40 -07:00
Alex Stokes
329bd4eb0f
Clean up peer ID tests 2019-08-14 09:00:40 -07:00
Alex Stokes
6506079a36
Generate peer IDs according to the spec 2019-08-14 09:00:39 -07:00
Alex Stokes
f9e859a8e9
Clean up peer.ID 2019-08-14 09:00:39 -07:00
Alex Stokes
c40314a043
Use new RSA key type 2019-08-14 09:00:39 -07:00
Alex Stokes
61f78c8feb
Add abstraction for a cryptographic key 2019-08-14 09:00:39 -07:00
Alex Stokes
2c68814bae
Add coincurve dep for secp256k1 keys 2019-08-14 09:00:39 -07:00
Alex Stokes
3736592a7a
Merge pull request #241 from ralexstokes/remove-codecov-ci
Remove code coverage check from CI
2019-08-14 08:58:24 -07:00
Alex Stokes
dc262ddb58
Remove badge from README 2019-08-14 08:49:38 -07:00
Alex Stokes
9bff7b8ebf
Remove code coverage check from CI 2019-08-13 21:25:22 -07:00
Alex Stokes
992585852f
Use explicit imports in top-level declaration 2019-08-13 17:43:21 -07:00
Alex Stokes
3debd2c808
Run black and isort w/ the new config 2019-08-13 14:36:42 -07:00
Alex Stokes
87375e0f23
Use the default line length for black ('no configuration')
and update `isort` to a stable configuration given the black line length
2019-08-13 14:36:05 -07:00
Alex Stokes
a937c6f3fa
Enable mypy in Makefile 2019-08-13 14:32:54 -07:00
Chih Cheng Liang
28f6de37ee
Fix the rest of the typing hints (#232)
* ignore kad

* fix swarm, and minor

* fix init and swarm

* ignore pb

* enable mypy

* fix basic host

* fix tcp

* fix mplex

* add typing for pb

* skip format pyi

* [mypy] no need to ignore pb now

* add typing to chat
2019-08-11 16:47:54 +08:00
Chih Cheng Liang
dbb702548f add NegotiableTransport type 2019-08-08 16:09:02 +08:00
Chih Cheng Liang
9851ee01fb sort import 2019-08-08 16:09:02 +08:00
Chih Cheng Liang
c536aa3e07 flake8 2019-08-08 16:09:02 +08:00
Chih Cheng Liang
5903012e0e add typing to protocol_muxer 2019-08-08 16:09:02 +08:00
NIC Lin
0d709364f8
Merge pull request #226 from NIC619/add_msg_validator
Add topic message validator
2019-08-07 12:10:45 +08:00
NIC619
a1dc68ab70
Apply PR feedback:
add validation failed test to `push_msg` test
2019-08-07 11:53:54 +08:00
NIC Lin
b26426214e
Update libp2p/pubsub/pubsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-08-07 11:43:32 +08:00
NIC619
d4febea469
Message was not enforced to carry signature yet 2019-08-06 13:05:31 +08:00
NIC619
1cea1264a4
Raise exception when topic validation failed 2019-08-06 12:38:31 +08:00
NIC619
9a1e5fe813
Add ValidationError 2019-08-06 12:37:34 +08:00
NIC619
47643a67c6
Apply PR feedback 2019-08-06 12:32:18 +08:00
NIC619
b96ef0e6c7
Fix:
`_is_subscribed_to_msg` need only subscribe to one of the topics
2019-08-05 18:20:04 +08:00
NIC619
a2efd03dfa
Schedule push_msg into a task 2019-08-05 18:19:32 +08:00
NIC619
2bb7f42c20
Add validators to push_msg 2019-08-05 18:19:32 +08:00
NIC619
19ce5bb420
Add signature_validator stub and docstring 2019-08-05 18:19:32 +08:00
NIC619
e1b86904e3
Add validate_msg and test 2019-08-05 18:19:32 +08:00
NIC619
ec2c566e5a
Fix validator return type and add docstring 2019-08-05 18:19:32 +08:00
NIC619
f8ca4fa1ef
Add get_msg_validators and test 2019-08-05 18:19:32 +08:00
NIC619
1ed14d0cc8
Add remove_topic_validator test 2019-08-05 18:19:32 +08:00
NIC619
cf69f7e800
Rename to set_topic_validator and add test 2019-08-05 18:19:31 +08:00
NIC619
b1f4813195
Add add/remove topic validator functions 2019-08-05 18:19:31 +08:00
NIC619
3973f1d13c
Add pubsub.topic_validators 2019-08-05 18:19:31 +08:00
NIC Lin
ccf4b62976
Merge pull request #203 from ChihChengLiang/tranport-typing
add typing to transport and stream_muxer
2019-08-05 17:19:11 +08:00
Chih Cheng Liang
cb3a59e0ac
ttl as int 2019-08-05 17:02:18 +08:00
Chih Cheng Liang
63c733c3f5
PR feedback 2019-08-05 16:58:34 +08:00
Chih Cheng Liang
ccfb6eb35f
remove constructor of TCP 2019-08-05 16:56:56 +08:00
Chih Cheng Liang
e763f57930
run isort 2019-08-05 11:47:23 +08:00
Chih Cheng Liang
7a04ebb51f
run black 2019-08-05 11:21:20 +08:00
Chih Cheng Liang
9e0a806218
move stream and connection interfaces to abc 2019-08-05 11:17:38 +08:00
Chih Cheng Liang
c804f5ad19
minor 2019-08-05 10:47:59 +08:00
Chih Cheng Liang
29091266fc
add still needed TYPE_CHECK 2019-08-05 10:46:49 +08:00
Chih Cheng Liang
87ef2e4618
remove if TYPE_CHECKING as much as possible 2019-08-05 10:46:49 +08:00
Chih Cheng Liang
4c9a930f84
stream_muxer done 2019-08-05 10:45:47 +08:00
Chih Cheng Liang
dadac423f2
typed muxed_connection_interface.py 2019-08-05 10:45:47 +08:00
Chih Cheng Liang
b64ed9fd6f
typed mplex.utils 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
239a5c88fb
add typing to mplex 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
36b7e8ded9
Refactor HeaderTags 2019-08-05 10:45:46 +08:00
Chih Cheng Liang
29fbb9e40a
add typing to transport 2019-08-05 10:42:43 +08:00
Alex Stokes
a20c172480 update isort line length 2019-08-04 12:37:41 +08:00
Alex Stokes
7477b29508 run black w/ extended line length 2019-08-04 12:37:41 +08:00
Alex Stokes
905dfa9a8d Remove dependence on make in tox CI run 2019-08-04 12:37:41 +08:00
Alex Stokes
251422a234 Match black line length to flake8 line length 2019-08-04 12:37:41 +08:00
Alex Stokes
cb301fcc51 Opt-out of linting on the special cases we have, given new ignore rules 2019-08-04 12:37:27 +08:00
Alex Stokes
03b1304fe7 Remove rules we want to enforce or are unnecessary with our style 2019-08-04 12:37:27 +08:00
Alex Stokes
a8acbb72c2
Merge pull request #223 from ralexstokes/remove-grpc
Remove `grpc` dependency
2019-08-03 12:28:38 -07:00
Alex Stokes
986da458e9
Directly use protobuf dep 2019-08-03 12:17:09 -07:00
Alex Stokes
69ec86c871
Remove grpc dependency 2019-08-03 11:17:28 -07:00
Alex Stokes
910c3fa6f1
Merge pull request #221 from mhchia/feature/add-flake8-and-change-cfg
Add flake8 and change flake8 configs
2019-08-03 11:13:58 -07:00
Alex Stokes
cf3904a56a
Merge pull request #207 from ralexstokes/update-chat-example
Update chat example
2019-08-03 09:49:51 -07:00
mhchia
0b11e32000
Remove the TODO flag for flake8 2019-08-04 00:32:32 +08:00
mhchia
1cc7e38846
Add flake8 to lint deps 2019-08-04 00:27:02 +08:00
mhchia
0a5b4a88ca
Fix flake8 for the existing code 2019-08-04 00:18:30 +08:00
mhchia
727342a767
Move flake8 settings to tox.ini
And add flake8 in the Makefile.
2019-08-04 00:11:49 +08:00
Arun Babu Neelicattu
f20b78c93f
Replace pylint with flake8
This change removes the use of pylint and implements similar
code style checks. This is complementary with black code formatter.
2019-08-04 00:06:34 +08:00
mhchia
4fef80595c Skip pb files in linters
In isort, black, and flake8, respectively.

Add `format` in Makefile

Run top level packages only

With `setup.py`, to avoid running other unrelated directories.

Refactor
2019-08-03 23:34:56 +08:00
Alex Stokes
c8005c8113 Run isort in repo 2019-08-03 17:50:14 +08:00
Alex Stokes
a92d933ed2 Add isort to CI check 2019-08-03 17:50:14 +08:00
Alex Stokes
d78e6dbf04 Add black-compatible isort config 2019-08-03 17:50:14 +08:00
Alex Stokes
201850397a Add isort as a lint dep 2019-08-03 17:50:14 +08:00
Alex Stokes
21e013e753
Merge pull request #214 from ralexstokes/remove-third-party-key-in-id
Refactor ID to not use third-party type for cryptographic keys
2019-08-02 22:50:53 -07:00
Alex Stokes
6090d2ca3b
Clean up old comments, print stmts, formatting, etc. 2019-08-02 21:58:31 -07:00
Alex Stokes
bd8d45fbc1
Refactor ID to not use third-party type for cryptographic keys
Remove `ID.from_privkey` which would require specific knowledge per cryptosystem
2019-08-02 21:51:16 -07:00
Alex Stokes
cff5fe0d5f
Add pytest config 2019-08-02 15:58:39 -07:00
Alex Stokes
28e1a03dc4
remove empty yamux impl 2019-08-02 15:03:59 -07:00
Alex Stokes
d47cddee24
Clean up extraneous files in tests 2019-08-02 14:54:40 -07:00
Alex Stokes
224b54ad93
Add basic flake8 config 2019-08-02 14:38:03 -07:00
Alex Stokes
2dfd7794b6
add note about protoc version and add Make command to generate protobuf files 2019-08-02 12:20:48 -07:00
Alex Stokes
e55d9f2e60
restore generated protobuf files 2019-08-02 12:09:27 -07:00
Alex Stokes
88a3a3159e
Add clarifying comment about InsecureTransport 2019-08-02 12:07:35 -07:00
Alex Stokes
b2367e35d8
Merge pull request #208 from ralexstokes/disable-mypy-ci
disable mypy check during CI temporarily and move `black --check` to Makefile
2019-08-02 12:01:47 -07:00
Alex Stokes
7b7c8ad30d
run black over dangling files 2019-08-02 11:47:08 -07:00
Alex Stokes
da9d5cadec
Disables mypy in CI
Also moves linting to Makefile to take advantage of globbing.
2019-08-02 11:46:43 -07:00
Alex Stokes
612330d318
Merge pull request #205 from mhchia/fix/remove-pylint-leftover
Remove pylint related lines and files
2019-08-02 10:50:44 -07:00
Alex Stokes
c8d175b373
Add a localhost option and fix the printed example to run another peer 2019-08-02 10:22:15 -07:00
Alex Stokes
430b4e2f89
Bail as soon as we know there is a port error 2019-08-02 10:21:41 -07:00
mhchia
2e94fcf56c
Remove pylint:disable 2019-08-02 23:26:06 +08:00
mhchia
06a9511ab4
Remove .pylintrc 2019-08-02 23:20:30 +08:00
NIC Lin
7a0fa7dd37
Merge pull request #196 from NIC619/fix_peer_id
Refactor `peer.ID` class and only takes in `bytes` type argument
2019-08-02 18:03:56 +08:00
NIC619
568454534f
Remove unused ID type conversions 2019-08-02 16:48:38 +08:00
NIC619
ee290b2ac2
Fix missing asterisk 2019-08-02 16:48:32 +08:00
NIC619
3a4b592a5a
Fix missing asterisk 2019-08-02 15:06:39 +08:00
Kevin Mai-Husan Chia
38423cc2a4
Merge pull request #200 from ChihChengLiang/add-typing-security
Add typing to security module
2019-08-02 14:57:24 +08:00
NIC619
0173b5e0d9
Add ID.to_base58() and comparison against str type 2019-08-02 14:46:01 +08:00
NIC619
924e965537
Change argument name of Network.listen and blakc format 2019-08-02 14:45:59 +08:00
NIC619
cd684aad9e
Update peer_id to type peer.ID in pubsub folder 2019-08-02 14:45:23 +08:00
NIC619
9562cb2a46
Rename:
`KadPeerInfo.peer_id` to `KadPeerInfo.peer_id_bytes`
2019-08-02 14:43:01 +08:00
NIC619
f00e80bc25
Fix wrong peer id type used in KadPeerInfo 2019-08-02 14:42:10 +08:00
NIC619
b928bdb356
Convert from base58/pubkey/privkey to class method 2019-08-02 14:42:10 +08:00
NIC619
80481252ca
Refactor ID to take in type bytes only 2019-08-02 14:42:10 +08:00
Chih Cheng Liang
10a8347c6a
PR feedback 2019-08-02 14:12:59 +08:00
Kevin Mai-Husan Chia
f6e456c96e
Merge pull request #198 from mhchia/fix/clean-up-tests-with-fixture
Use factories and fixtures in pubsub tests
2019-08-01 22:08:03 +08:00
mhchia
716c60ca6d
Reflect PR feedback
- Remove leftover `int` in `GossipsubParams`
- Remove default fields in tests
2019-08-01 21:38:14 +08:00
Chih Cheng Liang
e731f77f2d
minor 2019-08-01 19:13:43 +08:00
Chih Cheng Liang
a86f010c95
add typing to security 2019-08-01 19:12:11 +08:00
Chih Cheng Liang
030abcc959
add vscode 2019-08-01 17:55:15 +08:00
mhchia
33d233f5df
Add missing comma 2019-08-01 13:31:02 +08:00
mhchia
4eb846be7c
Remove the assertions for connect
Because it fails in `test_security_multistream.py` and this check is
indeed not necessary inside `connect`.(Should probably be outside the
function.)
2019-08-01 13:30:09 +08:00
mhchia
c72dfe1dd3
Use factories and fixtures in pubsub tests
Done
- Add factories using factory-boy
- Modify fixtures and tests to use factories
- Modify tests to use fixtures and factories
- Clean up
2019-08-01 13:30:08 +08:00
Kevin Mai-Husan Chia
9181cf95f0
Merge pull request #199 from ralexstokes/add-black-to-repo
Add `black` to repo
2019-08-01 10:53:44 +08:00
Alex Stokes
51cc710dc0
remove pylint 2019-07-31 15:14:56 -07:00
Alex Stokes
0ae9840928
Run black over repo 2019-07-31 15:00:12 -07:00
Alex Stokes
a2133d8c7c
Add black check to CI 2019-07-31 14:59:47 -07:00
Alex Stokes
936369aa5e
Add black as a lint dependency 2019-07-31 14:53:28 -07:00
Kevin Mai-Husan Chia
1727ba48d9
Merge pull request #190 from mhchia/feature/pubsub-test
Add `test_pubsub.py`
2019-07-31 16:40:38 +08:00
mhchia
ce369d47e9
Refine exception message 2019-07-31 16:23:07 +08:00
Kevin Mai-Husan Chia
c0e253a524
Update tests/pubsub/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-31 16:07:48 +08:00
Kevin Mai-Husan Chia
3bb63612a9
Update tests/pubsub/conftest.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-31 16:07:04 +08:00
mhchia
9683d5e8ac
Add tests for Pubsub
- `test_handle_subscription`
- `test_handle_talk`
- `test_message_all_peers`
2019-07-31 14:15:11 +08:00
mhchia
3a42d72cd9
Fix a minor bug for pb optional field
In `Pubsub.continuously_read_stream`, it checks whether this is a
control message enclosed in RPC message with `if rpc_incoming.control:`.
However, in pb2, the condition is always true because a default value is
returned when a field is not set. Solved it by changing it to
`if rpc_incoming.HasField("control"):`.
2019-07-31 14:15:11 +08:00
mhchia
037b95252d
Add tests for Pubsub
- `test_get_hello_packet`
- `test_continuously_read_stream`
- `test_publish`
- `test_push_msg`
2019-07-31 14:15:10 +08:00
mhchia
550289a439
Combine test_subscription.py to test_pubsub.py
And add a bunch of tests for pubsub
2019-07-31 14:15:10 +08:00
mhchia
96563c0d84
Add fixtures for pubsub and router
And a starting `test_pubsub.py`
2019-07-31 14:14:13 +08:00
NIC Lin
21e97407ef
Merge pull request #192 from NIC619/add_type_hint
Add type hints to host/, peer/, network/, pubsub/, routing/
2019-07-31 11:13:37 +08:00
NIC619
5e215901c0
Apply PR feedback 2019-07-30 23:41:28 +08:00
NIC619
76de01a17d
Add duplicate-code and cyclic-import to pylintrc 2019-07-30 18:01:01 +08:00
NIC619
437b7665c4
Fix:
type object not subscriptable
2019-07-30 18:00:30 +08:00
NIC619
60d6703964
Temporary disable pylint on tests folder 2019-07-30 17:41:46 +08:00
NIC619
c4105688d1
Fix after rebase 2019-07-30 17:31:08 +08:00
NIC619
e53727d301
Apply PR feedback: fix type hints 2019-07-30 16:28:25 +08:00
NIC619
2d4e23cfe2
Fix cyclic import and lint 2019-07-30 16:28:05 +08:00
NIC619
a4a0d79f6d
Improve import layout 2019-07-30 16:27:29 +08:00
NIC619
e7ac09cb94
Fix:
Add Gossipsub attribute `peers_protocol` and do cleanup when peer removed
2019-07-30 16:26:21 +08:00
NIC619
d716e90e17
Fix on type hints 2019-07-30 16:25:33 +08:00
NIC619
edd164c878
Add type hints to network folder 2019-07-30 16:25:17 +08:00
NIC619
e1592997a8
Add type hints to routing folder 2019-07-30 16:24:52 +08:00
NIC619
f2de986c74
Add type hints to peer folder 2019-07-30 16:24:51 +08:00
NIC619
b695b0e1ec
Add type hint to host folder 2019-07-30 16:24:34 +08:00
NIC619
b2f496d081
Fix type hints except pb msg in pubsub folder 2019-07-30 16:24:34 +08:00
NIC619
a0aa105867
Add type hint to pubsub notifee/interface 2019-07-30 16:23:15 +08:00
NIC619
3549f2ff8b
Add type hints to mcache.py 2019-07-30 16:20:48 +08:00
NIC619
63014eeaae
Add type hints to floodsub.py 2019-07-30 16:20:47 +08:00
NIC619
b920955db6
Add type hints to gossipsub.py 2019-07-30 15:39:50 +08:00
NIC619
8eb6a230ff
Fix and add type hints to pubsub.py 2019-07-30 15:32:58 +08:00
Kevin Mai-Husan Chia
f0046fa3e0
Merge pull request #187 from mhchia/feature/pubsub-publish
Add `Pubsub.publish`
2019-07-29 12:29:34 +08:00
mhchia
f02d38c0ee
Reflect PR feedback
* Rename `src` to `msg_forwarder` in pubsub/floodsub/gossipsub
* Rename Variables
* Sort imports
* Clean up
2019-07-29 12:09:35 +08:00
mhchia
74d831d4e2
Reflect PR feedback 2019-07-28 18:06:38 +08:00
Kevin Mai-Husan Chia
70c5c84f32
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:09:01 +08:00
Kevin Mai-Husan Chia
a1e20caebe
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:07:11 +08:00
Kevin Mai-Husan Chia
ffb3920468
Update libp2p/pubsub/floodsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:06:03 +08:00
Kevin Mai-Husan Chia
c252c62009
Update libp2p/pubsub/pubsub.py
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-28 16:05:29 +08:00
mhchia
766d8ba1e1
A little bit clean up 2019-07-27 12:06:36 +08:00
mhchia
c028aef2de
Fix all tests
- Dedup `perform_test_from_obj` and the test cases used in both
`test_floodsub` and `test_gossipsub_backward_compatibility.py`.
Therefore, they are put in the standalone file
`tests/pubsub/floodsub_integration_test_settings.py`. The functions
and testcases are imported from there then.
- IMO still need a refactor on the tests. There are still some duplicate
code.
2019-07-27 11:49:03 +08:00
mhchia
65aedcb25a
Fix several tests 2019-07-26 18:35:25 +08:00
mhchia
035d08b8bd
Fix test_floodsub.py 2019-07-25 23:11:27 +08:00
mhchia
dadcf8138e
Fix the tests according to pubsub.Publish
And refactored a bit.
2019-07-25 16:58:00 +08:00
mhchia
cae4f34034
Refactor floodsub.publish
Passed the first test of floodsub
2019-07-25 14:08:16 +08:00
mhchia
93cf5a2c32
A roughly skeleton of floodsub.publish
Still need to ensure when to deliver to ourselves
2019-07-24 22:33:32 +08:00
mhchia
3f52b0dc0a
Remove leftover imports 2019-07-24 21:57:46 +08:00
mhchia
b528c211b9
Temp modified publish 2019-07-24 21:55:57 +08:00
mhchia
218bdb42c4
Add basic functionalities of publish 2019-07-24 21:55:04 +08:00
mhchia
8f9e5a28ff
Temp 2019-07-24 21:54:36 +08:00
Kevin Mai-Husan Chia
f329c5a627
Merge pull request #188 from mhchia/feature/add-typing-for-pubusb
Add tox and mypy
2019-07-24 21:45:19 +08:00
mhchia
a97bac0a02
Remove sudo from .travis.yml 2019-07-24 21:30:23 +08:00
mhchia
d64c7d6d56
Add the missing type for port 2019-07-24 21:28:14 +08:00
mhchia
ecf4e373da
Remove unused aio_timers 2019-07-24 18:49:51 +08:00
mhchia
381f5ddc3a
Replace click with argparse 2019-07-24 18:43:49 +08:00
mhchia
04b7df9fcf
Lint examples in tox 2019-07-24 18:00:57 +08:00
mhchia
529829b9f1
Move codecov to tox.ini 2019-07-24 16:41:19 +08:00
mhchia
1ae306ae8f
Fix mypy command
- Remove requirements_dev.txt
- Add detailed versions
2019-07-24 16:34:55 +08:00
mhchia
d3a948be47
Fix error: Change params floodsub.publish back 2019-07-24 16:24:14 +08:00
mhchia
8da4032c3b
Let pylint not complain about FIXME, XXX 2019-07-24 16:23:30 +08:00
mhchia
e428897cc8
Add the missing tox in CI 2019-07-24 16:11:05 +08:00
mhchia
859ec6e241
Add the missing env for py37-test 2019-07-24 16:09:03 +08:00
mhchia
ae4c135ae1
Change travis CI config 2019-07-24 16:05:33 +08:00
mhchia
9497c3180f
Add tox
- Put extras_require to setup.py
- Add mypy
2019-07-24 15:54:30 +08:00
mhchia
d6c19e71a6
Add typing and notes in pubsub 2019-07-24 14:54:30 +08:00
NIC Lin
4819959e5a
Merge pull request #184 from NIC619/fix_handle_graft
Fix `gossipsub.handle_graft`
2019-07-24 13:55:07 +08:00
NIC619
b0b4ddd0ca
Fix test_handle_prune 2019-07-24 11:35:14 +08:00
NIC619
99eabe49eb
Add handle_prune test 2019-07-23 23:00:43 +08:00
NIC619
99252e49f8
Prevent re-adding peers to mesh 2019-07-23 23:00:43 +08:00
NIC619
42093e40ec
Add handle_graft test 2019-07-23 22:53:01 +08:00
NIC619
e52bfe3a51
Fix:
Respond GRAFT with PRUNE if not subscribed to the topic
2019-07-23 22:52:24 +08:00
NIC Lin
84824fd566
Merge pull request #182 from NIC619/fix_refactor_gossipsub_join
Fix and refactor: `gossipsub.join`
2019-07-23 18:12:37 +08:00
NIC619
0cc8a205fb
Fix test and reduce number of nodes in test_join 2019-07-23 17:28:46 +08:00
NIC619
c0a3af69e0
Apply PR feedback:
Check that node is in mesh peer's mesh after subscribe
2019-07-23 16:45:54 +08:00
NIC Lin
042e0ac916
Update tests/pubsub/test_gossipsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-07-23 16:37:41 +08:00
NIC Lin
3c3a9ac90b
Update tests/pubsub/test_gossipsub.py
Co-Authored-By: Kevin Mai-Husan Chia <mhchia@users.noreply.github.com>
2019-07-23 16:37:01 +08:00
NIC619
afc853a776
Apply PR feedback 2019-07-22 23:22:07 +08:00
NIC619
cdbeb63879
Add test 2019-07-22 19:28:12 +08:00
NIC619
67f9edb77d
Remove fanout topic after joining the topic 2019-07-22 19:28:07 +08:00
NIC619
4ab99485a6
Fix lint error 2019-07-21 23:32:54 +08:00
NIC619
14ee44c549
Lint test and add cleanup to the end 2019-07-21 23:16:42 +08:00
NIC619
41141c028b
FIx:
check topic exist in `pubsub.peer_topics`
2019-07-21 23:16:41 +08:00
NIC619
2c1c8dc8cf
Add gossipsub.join test 2019-07-21 23:16:41 +08:00
NIC619
fd1f318b0c
Fix:
in mesh heartbeat, select from gossipsub peers subscribed to the topic
2019-07-21 23:14:08 +08:00
NIC619
a26c7783d6
Add one_to_all_connect 2019-07-21 23:14:08 +08:00
NIC619
404dc67e83
Fix:
prevent selecting peers from topic not in peer topics
2019-07-21 23:14:08 +08:00
NIC619
b5c3420c16
Refactor gossipsub.join 2019-07-21 23:14:08 +08:00
NIC Lin
1e78c21eca
Merge pull request #181 from NIC619/fix_handle_unsubscribe
Fix: handle unsubscribe message
2019-07-21 20:09:51 +08:00
Kevin Mai-Husan Chia
69a35536f4
Merge pull request #179 from mhchia/feature/latest-maddr-and-pin-versions
Use the latest multiaddr and pin deps versions
2019-07-20 22:57:35 +08:00
mhchia
8d3b3fdb6c
Add debug msg 2019-07-20 22:47:37 +08:00
Kevin Mai-Husan Chia
085a5e7526
Apply suggestions from code review
Co-Authored-By: NIC Lin <twedusuck@gmail.com>
2019-07-20 22:43:32 +08:00
mhchia
a2d1aadf25
Add test for checking p2p code 2019-07-20 22:43:32 +08:00
mhchia
d3eb68fa50
Increase coverage 2019-07-20 22:43:32 +08:00
mhchia
73125b99b0
Fix the error due to the breaking change
In multiaddr, `split` is moved to `Multiaddr`'s method.
2019-07-20 22:43:31 +08:00
mhchia
0fbf45d8ca
Pin the versions
Especially, use the latest multiaddr
2019-07-20 22:42:18 +08:00
Kevin Mai-Husan Chia
4422888f6f
Merge pull request #180 from mhchia/fix/include-all-pkgs-setuptools
Change `packages` in `setup`
2019-07-20 22:39:57 +08:00
NIC Lin
d37f7f64e3
Merge pull request #176 from NIC619/small_fix_on_docstring
Fix some docstrings
2019-07-20 22:12:08 +08:00
NIC619
29aae7dca4
Add gossipsub join/leave test 2019-07-19 20:16:53 +08:00
NIC619
36575e8c9b
Add check to prevent gossipsub re-join and re-leave 2019-07-19 19:59:48 +08:00
NIC619
183eee0e85
Add self subscription tests 2019-07-18 13:39:22 +08:00
NIC619
89347be526
Prevent self re-unsubscription 2019-07-18 13:26:31 +08:00
NIC619
f25d97fbd3
Prevent self re-subscription 2019-07-15 16:32:05 +08:00
NIC619
6d9ec7a9c5
Handle the unsubscribe case 2019-07-15 16:28:29 +08:00
mhchia
edd02c498f
Ignore pb files 2019-07-11 20:58:03 +08:00
mhchia
31cd520076
Add the missing __init__.py in pb 2019-07-11 17:50:36 +08:00
mhchia
a7ce230d05
Change params multiaddr to maddr
To make pylint happy
2019-07-10 19:33:38 +08:00
mhchia
c8cb375d19
Make pylint happy
Not sure why it fails now, but happy previously.
2019-07-10 17:44:14 +08:00
mhchia
fa092505e8
Add missing "package file" 2019-07-10 00:07:58 +08:00
mhchia
10511d4930
Clean up a bit 2019-07-09 23:55:43 +08:00
mhchia
0188985918
Change packages in setup
Previously, only the top level `libp2p` is included.
This left only `__init__.py` in the built distribution.
Use `setuptools.find_packages` with `exclude` instead, to avoid tiringly
list all of the subpackages.
2019-07-09 23:29:03 +08:00
NIC619
900b6d9f37
Fix docstring 2019-06-21 11:51:11 -06:00
Jason Carver
a6f6078814
Add vim .swo files and .mypy_cache to .gitignore 2019-06-03 12:22:46 -07:00
stuckinaboot
b001256f5f
Merge pull request #166 from libp2p/peer-id
Full-length Peer ID
2019-05-22 12:58:45 -04:00
Stuckinaboot
9340d03660 Merge branch 'peer-id' of https://github.com/libp2p/py-libp2p into peer-id 2019-05-22 11:37:31 -04:00
Stuckinaboot
a4529d827d Modify peer_id str 2019-05-22 11:37:07 -04:00
Stuckinaboot
4b40339b63 Update peerid tests for longer peerids 2019-05-22 11:37:07 -04:00
Stuckinaboot
ae44c062df Remove truncating peerid when converting to string 2019-05-22 11:37:07 -04:00
stuckinaboot
8bdcc63296
Merge pull request #167 from ralexstokes/pin-multiaddr-version
Pin the version of multiaddr we require.
2019-05-22 11:33:49 -04:00
Stuckinaboot
f9721a2ade Modify peer_id str 2019-05-21 22:25:08 -04:00
Alex Stokes
c934100526
Pin the version of multiaddr we require.
Temporary workaround for https://github.com/multiformats/py-multiaddr/issues/47
2019-05-16 11:15:23 -04:00
Stuckinaboot
1250182e7c Update peerid tests for longer peerids 2019-05-08 20:19:13 -04:00
Stuckinaboot
702af198f2 Remove truncating peerid when converting to string 2019-05-08 20:19:02 -04:00
ZX
daec48b601
Merge pull request #165 from libp2p/green-apples
update green apples
2019-05-07 13:53:54 -04:00
zixuanzh
82682ab03f update green apples 2019-05-07 10:08:41 -04:00
ZX
49384ad3d7
Merge pull request #159 from libp2p/security-dev
Implement security
2019-05-07 09:56:19 -04:00
Stuckinaboot
867f3a70f6 Modify stream to be reader_writer 2019-05-07 01:31:54 -04:00
Robert Zajac
9052e8f8bd
The Gossipsub PR (#162)
* Add handle_rpc call to pubsub

* Scaffold gossipsub functions

* Add timer

* Implement most of mesh construction

* Implement emit and handle

* Implement fanout heartbeat

* Refactor emit

* some gossipsub cleanup and test

* minor lint stuff, more to come

* Implement publish

* Fix comment

* Modify pubsub/gossipsub so that floodsub tests pass using gossipsub router

* Add floodsub tests to gossipsub

* Handle case where select_from_minus, num_to_select > size(pool-minus)

* Add topic membership

* Implement handle ihave

* Implement most of iwant

* Add mcache.add and comments

* Refactor handle_ihave

* Implement stream write in handle_iwant

* Implement gossip heartbeat

* unresolved vars

* initial mcache code

* documenting mcache

* writing test/debugging mcache

* finished mcache test and debugged

* Make gossipsub backward compatibility its own file

* remove mcache prints

* DEBUGGING

* Add sender_peer_id to handle_rpc to get gossip test passing

* Modify gossipsub to make fanout work

* fanout maintenance test

* debugging gsub GOSSIP

* DEBUGGING

* debugged sender seen cachce

* adding lru, removing prints

* pylint cleanup

* Fix github comments in PR

* minor floodsub possible bugfix
2019-05-06 23:44:13 -04:00
Alex Haynes
eea6a9fda7
Merge pull request #157 from libp2p/router-refactor
Refactored RoutedHost into Injected Router
2019-05-05 14:45:22 -04:00
Alex Haynes
ea4fd8e81f requested changes 2019-05-05 14:32:41 -04:00
stuckinaboot
ff500e6d8e
Merge branch 'master' into security-dev 2019-05-01 17:26:23 -04:00
Stuckinaboot
144dc8d854 Move simple security to libp2p/security 2019-05-01 17:21:11 -04:00
Stuckinaboot
515a461172 Add more security tests 2019-05-01 17:13:01 -04:00
Jason Carver
81af1b6444
Merge pull request #26 from pipermerriam/piper/update-to-native-pypy-markdown-support
switch to native pypy markdown support
2019-05-01 13:53:19 -07:00
Piper Merriam
84d2b22a7b
switch to native pypy markdown support 2019-05-01 14:32:48 -06:00
Stuckinaboot
4333c2d061 Fix linting issues' 2019-05-01 13:54:19 -04:00
Alex Haynes
0f5854a83b
Merge pull request #160 from libp2p/providers
implement add get providers
2019-04-30 21:48:17 -04:00
Stuckinaboot
e555f17a7b Fix bug in security multistream 2019-04-30 16:07:26 -04:00
Stuckinaboot
a0bd6e5eb0 Add simple security with communication test 2019-04-30 03:27:06 -04:00
Stuckinaboot
f59f27d4d0 Integrate security selectin into libp2p system 2019-04-30 03:09:05 -04:00
Stuckinaboot
999e64854c Add security tests 2019-04-29 18:05:49 -04:00
Stuckinaboot
1fb4372ede Refine security 2019-04-29 18:05:38 -04:00
zixuanzh
e1d6fdae73 pass test 2019-04-28 18:18:56 -04:00
zixuanzh
db7be2d561 add simple test 2019-04-28 17:57:57 -04:00
Alex Haynes
2efc0d3b5c updated peerinfo encoding stored in DHT 2019-04-28 17:38:40 -04:00
zixuanzh
8ab387ac05 refactor add_router in swarm 2019-04-28 15:00:43 -04:00
ZX
21a99287b1
Merge branch 'master' into router-refactor 2019-04-28 14:51:54 -04:00
zixuanzh
3b1a3fb198 fix tests 2019-04-28 13:58:57 -04:00
Alex Haynes
b9ae73d317
Merge pull request #158 from libp2p/init-refactor
move transport to swarm constructor
2019-04-27 22:59:05 -04:00
Alex Haynes
fdaa4f56b9 working on adding tests 2019-04-27 21:59:25 -04:00
Stuckinaboot
eb26661652 Remove outdated encryption folder 2019-04-27 20:09:25 -04:00
Stuckinaboot
d861a00d60 Scaffold security 2019-04-27 19:42:05 -04:00
Christoph Burgdorf
720b2cf3d2 Add mypy support 2019-04-26 18:22:24 +02:00
zixuanzh
4436886371 implement add get providers 2019-04-25 13:25:09 -04:00
zixuanzh
35f75c4dcd move transport to swarm constructor 2019-04-24 22:36:09 -04:00
Alex Haynes
98f8df25ab Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:26:30 -04:00
Alex Haynes
159301306d Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:26:17 -04:00
Alex Haynes
2b8c7c294a Merge branch 'router-refactor' of https://github.com/libp2p/py-libp2p into router-refactor 2019-04-24 22:18:03 -04:00
Alex Haynes
7962060cc0 refactored routedhost into router passed to swarm 2019-04-24 22:17:35 -04:00
Alex Haynes
6c5bac53d7 refactored routedhost into router passed to swarm 2019-04-24 22:11:54 -04:00
Alex Haynes
fc4fc74b87
Merge pull request #153 from libp2p/routed-host
Added RoutedHost
2019-04-24 21:12:15 -04:00
Alex Haynes
c4f8bf6ce1 fixed requested changes 2019-04-24 20:59:22 -04:00
Alex Haynes
73ce3461e2 fixed typo 2019-04-21 13:45:18 -04:00
Alex Haynes
9c5cd05042 updated id access in kadpeerinfo 2019-04-21 13:27:44 -04:00
Alex Haynes
5fc57c1076 removed duplicate code 2019-04-21 13:22:11 -04:00
Robert Zajac
db7f890e93
remove unused umsgpack (#155) 2019-04-21 12:16:24 -04:00
Alex Haynes
e7424d3673 added RoutedHost and updated new_node to support it 2019-04-20 17:35:05 -04:00
ZX
367a939087
Merge pull request #152 from libp2p/node-refactor
Node refactor
2019-04-19 21:09:44 -04:00
Alex Haynes
b0edc47b9a removed commented out code 2019-04-19 21:04:18 -04:00
Alex Haynes
dbe3fcbf5a updated same_home_as to work for variable number of addresses 2019-04-19 21:00:55 -04:00
Alex Haynes
3fa6a165b3 Merge branch 'node-refactor' of https://github.com/libp2p/py-libp2p into node-refactor 2019-04-19 20:54:37 -04:00
zixuanzh
69f9aa5f0e refactor KadPeerInfo construction 2019-04-19 20:44:17 -04:00
zixuanzh
1512ae27a1 fix all tests 2019-04-19 20:09:32 -04:00
zixuanzh
fb687dad09 refactor peerinfo 2019-04-19 20:09:32 -04:00
zixuanzh
c03f2f63d2 replace id with peer_id 2019-04-19 20:09:32 -04:00
zixuanzh
9ddbd18ded replace node with KadPeerInfo
replace node with kadpeerinfo
2019-04-19 20:09:18 -04:00
Robert Zajac
5d4b4cf525
fixing chat (#151) 2019-04-18 15:56:02 -04:00
Alex Haynes
cce226c714
Merge pull request #129 from zaibon/peer_routing
[WIP] kademlia dht
2019-04-17 21:50:03 -04:00
Alex Haynes
1228b11bc9
Update test_basic.py 2019-04-17 21:44:45 -04:00
zixuanzh
9906e23b55 fixed linting 2019-04-17 21:39:30 -04:00
zixuanzh
002a46f0d4 add dependencies 2019-04-17 21:25:02 -04:00
ZX
14da7d49fe
Merge branch 'master' into peer_routing 2019-04-17 21:22:37 -04:00
zixuanzh
3c6ad886bb add dependencies 2019-04-17 21:21:17 -04:00
Alex Haynes
3ee75f0ecd updated travis config 2019-04-17 21:03:35 -04:00
Alex Haynes
d5c7cc7cb7 added changes from future commits to last passing bmuller commit 2019-04-17 20:21:59 -04:00
Robert Zajac
c8908f7e67
keep lru_dict in setup.py only 2019-04-09 20:50:21 -04:00
ZX
629d6dfd3f
Merge pull request #147 from libp2p/lru-cache
LRU cache on seen_messages #146
2019-04-06 12:26:15 -04:00
zixuanzh
7a298adc33 add simple lru test 2019-04-05 21:46:18 -04:00
zixuanzh
d04798ce7c lru cache seen_messages 2019-04-05 21:46:03 -04:00
zixuanzh
1f3aaded1d add lru-dict as requirement 2019-04-05 21:45:31 -04:00
stuckinaboot
ffd3bac2ef
Merge pull request #144 from libp2p/seqno
Seqno. Addresses seqno and removing unused print statements in #146
2019-04-05 17:50:47 -04:00
Stuckinaboot
686c55b09c Remove unnecessary print #146 2019-04-05 17:38:29 -04:00
Stuckinaboot
b4aa64bc39 Change >I to >Q for 64-bit big endian int 2019-04-05 17:30:35 -04:00
Stuckinaboot
3c0fca8979 Remove prints 2019-04-04 17:10:41 -04:00
Stuckinaboot
8221e1057c Merge branch 'seqno' of https://github.com/libp2p/py-libp2p into seqno 2019-04-04 17:09:51 -04:00
Stuckinaboot
22e503a260 Add test for ring topology multiple messages from two origins 2019-04-03 23:32:03 -04:00
Stuckinaboot
fc78928037 Add test for multiple messages from two origins 2019-04-03 23:32:03 -04:00
Stuckinaboot
211d6f6860 Add dummy node test 2019-04-03 23:32:03 -04:00
Stuckinaboot
feaa393c5f Fix seen messages bug 2019-04-03 23:32:03 -04:00
Stuckinaboot
9d16aa834d Modify pubsub to have seen message check incorporate seqno and node id 2019-04-03 23:32:03 -04:00
Stuckinaboot
c2b5389362 Adjust floodsub tests for new seqno util 2019-04-03 23:32:03 -04:00
Stuckinaboot
f6299c7dee Add priority queues to handle seqno 2019-04-03 23:32:03 -04:00
Stuckinaboot
e62b532c9c Merge branch 'seqno' of https://github.com/libp2p/py-libp2p into HEAD 2019-04-03 15:16:24 -04:00
Stuckinaboot
0fe21308ae Add test for ring topology multiple messages from two origins 2019-04-03 15:13:57 -04:00
Stuckinaboot
c270bda456 Add test for multiple messages from two origins 2019-04-03 15:13:57 -04:00
Stuckinaboot
3f2561e30b Add dummy node test 2019-04-03 15:13:57 -04:00
Stuckinaboot
b73b2f0841 Fix seen messages bug 2019-04-03 15:13:57 -04:00
Stuckinaboot
91dd3c0230 Modify pubsub to have seen message check incorporate seqno and node id 2019-04-03 15:13:57 -04:00
Stuckinaboot
a47b02dd26 Adjust floodsub tests for new seqno util 2019-04-03 15:13:56 -04:00
Stuckinaboot
502bb01242 Add priority queues to handle seqno 2019-04-03 15:13:56 -04:00
zixuanzh
7d3a8b5e77 clean up 2019-04-03 15:13:56 -04:00
zixuanzh
bc6a27a762 fix all tests 2019-04-03 15:13:56 -04:00
zixuanzh
372ed8336c remove message from test dummy 2019-04-03 15:13:56 -04:00
zixuanzh
73c1d87db1 update dummy account node 2019-04-03 15:13:56 -04:00
zixuanzh
fb5b3e4a24 reworked floodsub logic 2019-04-03 15:13:56 -04:00
zixuanzh
3836aa65f1 fix encoding issue 2019-04-03 15:13:56 -04:00
zixuanzh
65526a3319 remove Message from dummy account 2019-04-03 15:13:56 -04:00
zixuanzh
35a587f72a remove message.py 2019-04-03 15:13:56 -04:00
zixuanzh
c03ba881f2 reworked subscribe unsubsrcibe 2019-04-03 15:13:56 -04:00
Alex Haynes
83de4f2972 RPC conversion progress 2019-04-03 15:13:56 -04:00
zixuanzh
dea31531da update dependencies 2019-04-03 15:13:56 -04:00
zixuanzh
a25aa0b034 rewrote get_hello_packet 2019-04-03 15:13:56 -04:00
zixuanzh
f1a20f12dc update from to from_id in proto 2019-04-03 15:13:56 -04:00
zixuanzh
2aa24cbfe8 add generated rpc code 2019-04-03 15:13:56 -04:00
zixuanzh
f655148d9c add pubsub proto 2019-04-03 15:13:56 -04:00
Alex Haynes
0d7b8c3e5d added rpc.proto from go repo 2019-04-03 15:13:56 -04:00
stuckinaboot
9c6d441f9f [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-04-03 15:13:56 -04:00
zixuanzh
fee905ace2 fix linting issues 2019-04-03 15:13:56 -04:00
zixuanzh
2e437e5b8b add test for listen event 2019-04-03 15:13:56 -04:00
zixuanzh
2cb0279b57 fix existing tests 2019-04-03 15:13:56 -04:00
zixuanzh
4c89a59406 refactor host setup helper 2019-04-03 15:13:56 -04:00
zixuanzh
fca5782d01 refactor new_node 2019-04-03 15:13:56 -04:00
Stuckinaboot
ed3c5f4b51 Add additional initiator and non-initiator notifee tests 2019-04-03 15:13:56 -04:00
Stuckinaboot
dfe2b0e7e7 Add return value to Notify 2019-04-03 15:13:56 -04:00
Stuckinaboot
4b9327c5a3 Add opened_stream call for non-initiator 2019-04-03 15:13:56 -04:00
Stuckinaboot
a0ef64fff7 Fix outstanding unrelated lint issue in multiselect_client 2019-04-03 15:13:56 -04:00
Stuckinaboot
92bfd816b9 Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot
5cabc11687 Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot
f58afc0eac Fix linting issues 2019-04-03 15:13:56 -04:00
Stuckinaboot
fa97b97748 Refactor test setup to remove duplicate code 2019-04-03 15:13:56 -04:00
Stuckinaboot
86ce7530d5 Add conn to net stream and conn tests 2019-04-03 15:13:56 -04:00
Stuckinaboot
66703bd414 Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot
4298b0631f Fix linting issue 2019-04-03 15:13:56 -04:00
Stuckinaboot
1ecdcf7df4 Make notifee functions all async 2019-04-03 15:13:56 -04:00
Stuckinaboot
3beec9d4ae Add tests for notify 2019-04-03 15:13:56 -04:00
Stuckinaboot
c465141331 Implement notify feature 2019-04-03 15:13:56 -04:00
Stuckinaboot
1701fba42e Add notify function to network interface 2019-04-03 15:13:56 -04:00
Stuckinaboot
204c103d92 Add notifee interface 2019-04-03 15:13:56 -04:00
Stuckinaboot
005ea67e82 Merge branch 'seqno' of https://github.com/libp2p/py-libp2p into seqno 2019-04-03 14:45:51 -04:00
Stuckinaboot
0713cddbd1 Add test for ring topology multiple messages from two origins 2019-04-03 14:24:40 -04:00
Stuckinaboot
db53976a6a Add test for multiple messages from two origins 2019-04-03 14:24:40 -04:00
Stuckinaboot
62d1bff888 Add dummy node test 2019-04-03 14:24:40 -04:00
Stuckinaboot
60fe42bf60 Fix seen messages bug 2019-04-03 14:24:40 -04:00
Stuckinaboot
547e7b3546 Modify pubsub to have seen message check incorporate seqno and node id 2019-04-03 14:24:40 -04:00
Stuckinaboot
2934f93fc9 Adjust floodsub tests for new seqno util 2019-04-03 14:24:40 -04:00
Stuckinaboot
56fbdf3f2f Add priority queues to handle seqno 2019-04-03 14:24:40 -04:00
zixuanzh
0de028a236 clean up 2019-04-03 14:24:40 -04:00
zixuanzh
f50ba04178 fix all tests 2019-04-03 14:24:40 -04:00
zixuanzh
f287c73236 remove message from test dummy 2019-04-03 14:24:40 -04:00
zixuanzh
974cb0b06f update dummy account node 2019-04-03 14:24:40 -04:00
zixuanzh
87269a9524 reworked floodsub logic 2019-04-03 14:24:40 -04:00
zixuanzh
1cbd909dc2 fix encoding issue 2019-04-03 14:24:40 -04:00
zixuanzh
a2e22159b2 remove Message from dummy account 2019-04-03 14:24:40 -04:00
zixuanzh
2183bada41 remove message.py 2019-04-03 14:24:40 -04:00
zixuanzh
598b6b257f reworked subscribe unsubsrcibe 2019-04-03 14:24:40 -04:00
Alex Haynes
75c12d4aed RPC conversion progress 2019-04-03 14:24:40 -04:00
zixuanzh
adce7f0a05 update dependencies 2019-04-03 14:24:40 -04:00
zixuanzh
b297aa9576 rewrote get_hello_packet 2019-04-03 14:24:40 -04:00
zixuanzh
2b41818561 update from to from_id in proto 2019-04-03 14:24:40 -04:00
zixuanzh
f2de3735ee add generated rpc code 2019-04-03 14:24:40 -04:00
zixuanzh
2aa7c4a106 add pubsub proto 2019-04-03 14:24:40 -04:00
Alex Haynes
1c08e9e55c added rpc.proto from go repo 2019-04-03 14:24:39 -04:00
stuckinaboot
8d0f40a378 [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-04-03 14:24:39 -04:00
zixuanzh
fa53c5a866 fix linting issues 2019-04-03 14:24:39 -04:00
zixuanzh
cc7c08bac4 add test for listen event 2019-04-03 14:24:39 -04:00
zixuanzh
3e0d6ea126 fix existing tests 2019-04-03 14:24:39 -04:00
zixuanzh
9aa41b106a refactor host setup helper 2019-04-03 14:24:39 -04:00
zixuanzh
474602c4f7 refactor new_node 2019-04-03 14:24:39 -04:00
Stuckinaboot
51cd9c8e56 Add additional initiator and non-initiator notifee tests 2019-04-03 14:24:39 -04:00
Stuckinaboot
7d45131f37 Add return value to Notify 2019-04-03 14:24:39 -04:00
Stuckinaboot
d800b3ef41 Add opened_stream call for non-initiator 2019-04-03 14:24:39 -04:00
Stuckinaboot
1384cebf9c Fix outstanding unrelated lint issue in multiselect_client 2019-04-03 14:24:39 -04:00
Stuckinaboot
ce22f06773 Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot
ef827f90ad Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot
15883fcf09 Fix linting issues 2019-04-03 14:24:39 -04:00
Stuckinaboot
b340b89819 Refactor test setup to remove duplicate code 2019-04-03 14:24:39 -04:00
Stuckinaboot
91b4d66d50 Add conn to net stream and conn tests 2019-04-03 14:24:39 -04:00
Stuckinaboot
86b4bb5e55 Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot
a1c0165484 Fix linting issue 2019-04-03 14:24:39 -04:00
Stuckinaboot
0d2aa3f5b1 Make notifee functions all async 2019-04-03 14:24:39 -04:00
Stuckinaboot
5a4af0c81c Add tests for notify 2019-04-03 14:24:39 -04:00
Stuckinaboot
3dbb969b0b Implement notify feature 2019-04-03 14:24:39 -04:00
Stuckinaboot
95f1f0bbf0 Add notify function to network interface 2019-04-03 14:24:39 -04:00
Stuckinaboot
510a5eaa34 Add notifee interface 2019-04-03 14:24:39 -04:00
Stuckinaboot
c61f4297b0 Add test for ring topology multiple messages from two origins 2019-04-03 14:20:50 -04:00
Stuckinaboot
0ff155660d Add test for multiple messages from two origins 2019-04-03 14:17:33 -04:00
ZX
0190481e10
Merge pull request #141 from libp2p/floodsub-rpc
Floodsub RPC
2019-04-03 14:15:32 -04:00
zixuanzh
225bd390df add source to rpc.proto 2019-04-03 14:08:05 -04:00
zixuanzh
3a52d29cb7 remove redundant proto file 2019-04-03 14:05:37 -04:00
David Sanders
3c1bbfe738
Merge pull request #24 from davesque/updates
Update default author email
2019-04-03 11:55:57 -06:00
Stuckinaboot
e6605f22e9 Add dummy node test 2019-04-03 00:35:39 -04:00
Stuckinaboot
583b79fad7 Fix seen messages bug 2019-04-03 00:34:39 -04:00
Stuckinaboot
726d083a3a Modify pubsub to have seen message check incorporate seqno and node id 2019-04-02 22:34:01 -04:00
Stuckinaboot
1296589800 Adjust floodsub tests for new seqno util 2019-04-02 22:05:32 -04:00
Stuckinaboot
71282678c4 Add priority queues to handle seqno 2019-04-02 22:05:14 -04:00
zixuanzh
0238dff217 remove unused code 2019-04-02 21:17:48 -04:00
zixuanzh
41d1aae55b clean up 2019-04-01 16:55:44 -04:00
zixuanzh
6eb070b78e fix all tests 2019-04-01 16:23:20 -04:00
zixuanzh
2e5e7e3c10 remove message from test dummy 2019-04-01 15:04:20 -04:00
zixuanzh
de6bc011f0 update dummy account node 2019-03-31 22:16:49 -04:00
zixuanzh
89a19a9213 reworked floodsub logic 2019-03-31 22:16:28 -04:00
zixuanzh
971dbe1a96 fix encoding issue 2019-03-30 19:30:58 -04:00
zixuanzh
ec7bc45a58 remove Message from dummy account 2019-03-30 19:12:31 -04:00
zixuanzh
f5af4b9016 remove message.py 2019-03-30 18:49:50 -04:00
zixuanzh
aec783b843 reworked subscribe unsubsrcibe 2019-03-30 17:59:08 -04:00
Alex Haynes
bf17f424b3 RPC conversion progress 2019-03-29 16:23:30 -04:00
Alex Haynes
9be1214c44 Merge branch 'floodsub-rpc' of https://github.com/libp2p/py-libp2p into floodsub-rpc 2019-03-29 15:02:15 -04:00
zixuanzh
a81628f99d update dependencies 2019-03-28 15:42:55 -04:00
zixuanzh
3cfeccaf17 rewrote get_hello_packet 2019-03-28 15:25:33 -04:00
zixuanzh
81d121a029 update from to from_id in proto 2019-03-28 15:23:56 -04:00
zixuanzh
1e8d93fcf6 add generated rpc code 2019-03-28 09:55:14 -04:00
zixuanzh
041e0fbe34 add pubsub proto 2019-03-28 09:41:38 -04:00
Christophe de Carvalho
c5289952ee
add routing interfaces 2019-03-26 20:44:01 +01:00
Christophe de Carvalho
2d1b9a03d1
clean up 2019-03-26 20:20:14 +01:00
Christophe de Carvalho
cd8cb5c443
refactoring of the code to implement IAdvertiser and IDiscoverer 2019-03-26 20:16:16 +01:00
Christophe de Carvalho Pereira Martins
4889a0a790
First POC of peer routing using kademlia lib 2019-03-26 20:15:19 +01:00
Christophe de Carvalho Pereira Martins
c24a279d2d
update kadmelia lib 2019-03-26 20:14:42 +01:00
David Sanders
5e6dc29991
Update default author email 2019-03-26 12:11:50 -06:00
David Sanders
0b85056ca8
Merge pull request #23 from davesque/travis
Remove travis config
2019-03-26 11:49:07 -06:00
David Sanders
5f3e6b37bf
Remove travis config 2019-03-25 20:04:26 -06:00
David Sanders
7bffe4b08c
Merge pull request #22 from davesque/updates
Remove attribution to specific person, update README
2019-03-25 17:52:00 -06:00
David Sanders
de4bdf9e30
Remove attribution to specific person 2019-03-25 15:14:04 -06:00
Alex Haynes
cb6b76d3b5 added rpc.proto from go repo 2019-03-24 15:08:36 -04:00
stuckinaboot
57077cd3b4 [WIP] PubSub and FloodSub development (#133)
* Add notifee interface

* Add notify function to network interface

* Implement notify feature

* Add tests for notify

* Make notifee functions all async

* Fix linting issue

* Fix linting issue

* Scaffold pubsub router interface

* Scaffold pubsub directory

* Store peer_id in muxed connection

* Implement pubsub notifee

* Remove outdated files

* Implement pubsub first attempt

* Prepare pubsub for floodsub

* Add mplex conn to net stream and add conn in notify tests

* Implement floodsub

* Use NetStream in generic protocol handler

* Debugging async issues

* Modify test to perform proper assert. Test passes

* Remove callbacks. Reduce sleep time

* Add simple three node test

* Clean up code. Add message classes

* Add test for two topics

* Add conn to net stream and conn tests

* Refactor test setup to remove duplicate code

* Fix linting issues

* Fix linting issue

* Fix linting issue

* Fix outstanding unrelated lint issue in multiselect_client

* Add connect function

* Remove debug prints

* Remove debug prints from floodsub

* Use MessageTalk in place of direct message breakdown

* Remove extra prints

* Remove outdated function

* Add message to queues for all topics in message

* Debugging

* Add message self delivery

* Increase read timeout to 5 to get pubsub tests passing

* Refactor testing helper func. Add tests

* Add tests and increase timeout to get tests passing

* Add dummy account demo scaffolding

* Attempt to use threads. Test fails

* Implement basic dummy node tests using threads

* Add generic testing function

* Add simple seven node tree test

* Add more complex seven node tree tests

* Add five node ring tests

* Remove unnecessary get_message_type func

* Add documentation to classes

* Add message id to messages

* Add documentation to test helper func

* Add docs to dummy account node helper func

* Add more docs to dummy account node test helper func

* fixed linting errors in floodsub

* small notify bugfix

* move pubsub into libp2p

* fixed pubsub linting

* fixing pubsub test failures

* linting
2019-03-23 13:52:02 -04:00
ZX
6c4bbd1e85
Merge pull request #139 from libp2p/swarm-refactor
Refactor libp2p new_node creation
2019-03-19 00:03:24 -04:00
zixuanzh
d67aeb1974 fix linting issues 2019-03-17 21:30:56 -04:00
zixuanzh
0d15126ffc add test for listen event 2019-03-17 21:15:14 -04:00
zixuanzh
1886258fbf fix existing tests 2019-03-17 19:33:40 -04:00
zixuanzh
72a8d55faa refactor host setup helper 2019-03-17 19:33:10 -04:00
zixuanzh
fea26d9087 refactor new_node 2019-03-17 19:20:46 -04:00
Stuckinaboot
d42a0d7e9f Add additional initiator and non-initiator notifee tests 2019-03-15 11:22:20 -04:00
Stuckinaboot
a5fa053815 Add return value to Notify 2019-03-15 11:22:20 -04:00
Stuckinaboot
01099e9f2c Add opened_stream call for non-initiator 2019-03-15 11:22:20 -04:00
Stuckinaboot
745c5e677c Fix outstanding unrelated lint issue in multiselect_client 2019-03-15 11:22:20 -04:00
Stuckinaboot
58bdee499b Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot
a46779918f Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot
0c60e28150 Fix linting issues 2019-03-15 11:22:20 -04:00
Stuckinaboot
2eb8382280 Refactor test setup to remove duplicate code 2019-03-15 11:22:20 -04:00
Stuckinaboot
22d6c97ef3 Add conn to net stream and conn tests 2019-03-15 11:22:20 -04:00
Stuckinaboot
26dd385c95 Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot
670076035c Fix linting issue 2019-03-15 11:22:20 -04:00
Stuckinaboot
3e64520506 Make notifee functions all async 2019-03-15 11:22:20 -04:00
Stuckinaboot
7a8878411f Add tests for notify 2019-03-15 11:22:20 -04:00
Stuckinaboot
1ce8225c7d Implement notify feature 2019-03-15 11:22:20 -04:00
Stuckinaboot
f7d1bb3a89 Add notify function to network interface 2019-03-15 11:22:20 -04:00
Stuckinaboot
cdfcdd528e Add notifee interface 2019-03-15 11:22:20 -04:00
Hyungsuk Kang
78d0a658bb Typo fixes (#127)
* Update transport_interface.py

* Update tcp.py

* Update README.md
2019-02-24 23:11:54 -05:00
Robert Zajac
82840b5e6c
Stream rearchitecture (#126)
* Add generic protocol handler

* Add generic protocol handler to stream muxing pipeline

* Modify conn_handler to only deal with connections

* mplex accept stream architecture changes

* Add create generic protocol handler

* Fix minor bugs

* who would win 4 devs or one not

* Debugging

* rearch with handle_incoming infinite loop, seems to work, needs cleanup"

* passing linting, still needs cleanup

* fixing linting again; code still needs cleanup

* fixing tests; code still needs cleanup

* adding test cleanup and task cleanup, removing prints

* linting, and cleanup complete

* storing connections based on peer id

* remove dead code

* remove unnecessary peer_id
2019-02-24 20:58:23 -05:00
Alex Haynes
17c778de15
Peer Discovery Interface (#123)
* added file

* basic interface modeled on go repo

* fixed linting

* updated based on comments
2019-02-24 18:37:56 -05:00
Robert Zajac
e3e3ac30b1
Adding sponsorship ack to EF (#125)
Now that sponsorship has been announced (see link in blog), we can add acknowledgement and close out #60.
2019-02-22 15:02:17 -05:00
Alex Haynes
934714df62 updated readme (#124) 2019-02-12 10:56:03 -05:00
Robert Zajac
e34603c364
fixing chat to use public IP (#122) 2019-02-10 20:52:05 -05:00
David Sanders
2fb32d204d
Merge pull request #21 from davesque/pydocstyle
Add docstring checking with pydocstyle
2019-01-25 10:00:24 -07:00
David Sanders
dd74824840
Add docstring checking with pydocstyle 2019-01-24 13:57:17 -07:00
Jason Carver
66e6b7e3c4
Merge pull request #19 from ethereum/drop-py35
Drop py3.5, add py3.7
2019-01-16 13:55:56 -08:00
Jason Carver
caf9050a19 Drop py3.5, add py3.7 2019-01-15 16:06:18 -08:00
Jason Carver
03f00d3fbe
Merge pull request #12 from njgheorghita/update-bumpversion
[WIP] Add changelog instructions
2018-09-05 11:00:43 -07:00
Nick Gheorghita
0b004a8597
Add changelog instructions 2018-09-04 18:46:05 -06:00
Jason Carver
5ad0973c3d
Ignore any internal _utils module 2018-08-29 09:47:22 -07:00
Jason Carver
56e4b20080
Merge pull request #7 from davesque/fix-readlink
Make template filler more BSD friendly
2018-06-07 12:04:01 -07:00
Jason Carver
af727b97f2
Merge pull request #8 from davesque/twine
Use twine for pypi uploading per packaging docs
2018-06-07 12:02:28 -07:00
Jason Carver
e0af03a299
Merge pull request #10 from davesque/eth-utils-version
Update eth-utils version requirement
2018-06-07 11:52:59 -07:00
Jason Carver
24d6899caa
Merge pull request #9 from davesque/gpg-sign-setting
Fix ineffectual commands in Makefile
2018-06-07 11:52:30 -07:00
David Sanders
d3537cf1ab
Update eth-utils version requirement 2018-06-07 12:27:07 -06:00
David Sanders
a9d531615d
Fix ineffectual commands in Makefile 2018-06-07 12:23:32 -06:00
David Sanders
016f48d17d
Use twine for pypi uploading per packaging docs
See here:
https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives
2018-06-06 19:35:31 -06:00
David Sanders
6ed7bb011f
Make template filler more BSD friendly 2018-06-06 18:49:12 -06:00
Jason Carver
83ce66097c
Merge pull request #6 from davesque/isort-config
Fix possibly incorrect isort config
2018-06-06 11:32:12 -07:00
David Sanders
0088da3781
Fix possibly incorrect isort config 2018-06-04 18:40:58 -06:00
Jason Carver
28787fe9f6
Merge pull request #5 from davesque/misc-fixes
Misc fixes
2018-06-04 16:45:32 -06:00
David Sanders
0a83658b3e
Lint in setup.py 2018-06-04 16:37:52 -06:00
David Sanders
e7d0a8577b
Fix testing dependencies 2018-06-04 16:37:38 -06:00
David Sanders
00819dd924
Whitespace 2018-06-04 16:36:35 -06:00
David Sanders
32c8c981f2
Remove references to deprecated "py.test" 2018-06-04 16:36:02 -06:00
Jason Carver
8352ad5afa
Merge pull request #4 from davesque/readme-updates
Some readme updates
2018-06-01 14:14:56 -06:00
David Sanders
5e509100e7
Some readme updates 2018-06-01 14:05:57 -06:00
Jason Carver
ac85506912
make merge script executable 2018-05-15 13:02:56 -07:00
Jason Carver
b357fda082 refill from any directory 2018-05-15 12:33:36 -07:00
Jason Carver
71ba221c49 Enable template refilling 2018-05-15 12:29:21 -07:00
Jason Carver
85e33cf380 move template filler to own directory 2018-05-15 12:17:36 -07:00
Jason Carver
1091971637 remove old merge code from circle template 2018-05-15 12:09:51 -07:00
Jason Carver
c18b0cecfc
Merge pull request #3 from carver/circle-merge
circleci: merge (and reattempt) PR before testing
2018-05-15 12:02:29 -07:00
Jason Carver
8815766abf circleci: merge (and reattempt) PR before testing 2018-05-15 11:54:33 -07:00
Jason Carver
1f89c6385a fix gitignore typo about internals module in docs/ 2018-04-25 18:50:01 -07:00
Jason Carver
285f5c6c70 Remove duplicate python version mention in readme
Rely on the badge instead
2018-04-25 15:39:52 -07:00
Jason Carver
b98147efca keep template filler around for future merges 2018-04-25 15:20:03 -07:00
Jason Carver
0dd03f0a3f re-add MODULE_NAME in new template filler 2018-04-25 15:19:37 -07:00
Jason Carver
d7ce1100ed gitignore utils and internals docs; fill-in fixup 2018-04-25 15:05:30 -07:00
Jason Carver
5a9574a72c
Merge pull request #2 from davesque/updates
Couple more updates
2018-04-23 14:08:04 -07:00
David Sanders
cdc83bd498
Make find replace script more macOS friendly
Also, make handling of arbitrary filenames a bit more robust.
2018-04-22 01:35:08 -06:00
David Sanders
f7535eb992
Add circle ci status badge 2018-04-22 01:34:42 -06:00
Jason Carver
43960ffb5e
Merge pull request #1 from davesque/circleci
Make Circle CI test against merge with PR base
2018-04-19 16:16:38 -07:00
David Sanders
30473b3ff5
Make Circle CI test against merge with PR base 2018-04-19 16:31:48 -06:00
Jason Carver
84ecb7effc require python 3.5 during setup (and reject py4) 2018-04-10 10:16:17 -07:00
Jason Carver
031b1175f4 force tox to rebuild in circle runs 2018-02-28 14:20:39 -08:00
Jason Carver
588b1af6ee Makefile: deploy to upstream 2018-02-28 10:29:45 -08:00
Jason Carver
974fae3a2b travis config: use latest pypy3 container 2018-02-20 18:10:30 -08:00
Jason Carver
f0e0b10cc5 add pypy3 as a default test environment in circle 2018-02-20 18:09:58 -08:00
Jason Carver
b3461e9c93 tox.ini bugfix: whitelist make 2018-02-07 14:40:01 -08:00
Jason Carver
58902032c7 add doctest run to circle-ci 2018-02-07 14:37:30 -08:00
Jason Carver
60b026e3ed add doctest run to travis 2018-02-07 11:40:35 -08:00
Jason Carver
ecb3731cbe can't override python that way in tox 2018-02-07 11:26:47 -08:00
Jason Carver
37d978c5b3 squash warning about using make inside tox 2018-02-07 11:24:37 -08:00
Jason Carver
a1ba89ed41 every tox environment should have a basic python 2018-02-07 11:18:47 -08:00
Jason Carver
a9d9fec258 add doctest by default 2018-02-07 11:14:11 -08:00
Jason Carver
beb1b10ee8 silence flake8 error during module import test 2018-02-06 16:10:25 -08:00
Jason Carver
1695a3b326 bugfix: setuptools reference to pypy 2018-02-06 16:04:41 -08:00
Jason Carver
6abf066779 add module import test 2018-02-06 15:56:11 -08:00
Jason Carver
7bfa2ac5d4 add pypy3 as a supported environment, by default 2018-02-06 15:55:49 -08:00
Jason Carver
b825c36def docs: show badge, link to release notes 2018-02-06 15:42:21 -08:00
Jason Carver
4bb4f720db add badges: pypi version and pythons supported 2018-02-06 15:33:04 -08:00
Jason Carver
f7b0f07b45 fixups: rm template filler, xfail strict, make bug 2018-02-01 13:07:33 -08:00
Jason Carver
cb71fb4430 add circle ci config 2018-02-01 09:42:48 -08:00
Jason Carver
9a7d72a816 pytest default settings, plus pytest-watch 2018-01-30 15:38:42 -08:00
Jason Carver
0e57d8e262 when do you not want ipython in local dev? never 2018-01-30 15:18:59 -08:00
Jason Carver
96a371705b bugfix: run tox -elint *after* isort
otherwise tox fails, and the auto-lint never runs
2018-01-30 15:17:50 -08:00
Jason Carver
109a0866ed requirements.txt: best way to build on readthedocs 2018-01-30 15:17:03 -08:00
Jason Carver
195249cdab docs cleanup: Makefile, conf, warnings 2018-01-30 11:18:16 -08:00
Jason Carver
76bfefa66c add make lint-roll to locally auto de-lint 2018-01-26 19:01:36 -08:00
Jason Carver
2938497f02 Link to dev tactical manual 2018-01-25 17:21:42 -08:00
Jason Carver
3290919ff1 add github templates for issues and pull requests 2018-01-25 15:51:14 -08:00
Jason Carver
1d4e19d929 add docs index 2018-01-24 17:07:48 -08:00
Jason Carver
ca670f43a4 start at devnum 0 so the first bump goes to 1 2018-01-24 16:35:36 -08:00
Jason Carver
4a8b7b06af shorten extra name for documentation install 2018-01-24 16:33:03 -08:00
Jason Carver
9cdfd649d5 create empty module with supplied name 2018-01-24 16:31:02 -08:00
Jason Carver
cdba56a295 skip replacing variables inside .git/* files 2018-01-24 16:24:31 -08:00
Jason Carver
3d87ff7153 Replace all template vars 2018-01-24 16:24:08 -08:00
Jason Carver
a7955a560e init 2018-01-24 16:00:28 -08:00
270 changed files with 18060 additions and 3251 deletions

23
.bumpversion.cfg Normal file
View File

@ -0,0 +1,23 @@
[bumpversion]
current_version = 0.1.5
commit = True
tag = True
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(-(?P<stage>[^.]*)\.(?P<devnum>\d+))?
serialize =
{major}.{minor}.{patch}-{stage}.{devnum}
{major}.{minor}.{patch}
[bumpversion:part:stage]
optional_value = stable
first_value = stable
values =
alpha
beta
stable
[bumpversion:part:devnum]
[bumpversion:file:setup.py]
search = version="{current_version}",
replace = version="{new_version}",

77
.circleci/config.yml Normal file
View File

@ -0,0 +1,77 @@
version: 2.0
# heavily inspired by https://raw.githubusercontent.com/pinax/pinax-wiki/6bd2a99ab6f702e300d708532a6d1d9aa638b9f8/.circleci/config.yml
common: &common
working_directory: ~/repo
steps:
- checkout
- run:
name: merge pull request base
command: ./.circleci/merge_pr.sh
- run:
name: merge pull request base (2nd try)
command: ./.circleci/merge_pr.sh
when: on_fail
- run:
name: merge pull request base (3nd try)
command: ./.circleci/merge_pr.sh
when: on_fail
- restore_cache:
keys:
- cache-{{ .Environment.CIRCLE_JOB }}-{{ checksum "setup.py" }}-{{ checksum "tox.ini" }}
- run:
name: install dependencies
command: pip install --user tox
- run:
name: run tox
command: ~/.local/bin/tox -r
- save_cache:
paths:
- .hypothesis
- .tox
- ~/.cache/pip
- ~/.local
- ./eggs
key: cache-{{ .Environment.CIRCLE_JOB }}-{{ checksum "setup.py" }}-{{ checksum "tox.ini" }}
jobs:
docs:
<<: *common
docker:
- image: circleci/python:3.6
environment:
TOXENV: docs
lint:
<<: *common
docker:
- image: circleci/python:3.6
environment:
TOXENV: lint
py36-core:
<<: *common
docker:
- image: circleci/python:3.6
environment:
TOXENV: py36-core
py37-core:
<<: *common
docker:
- image: circleci/python:3.7
environment:
TOXENV: py37-core
pypy3-core:
<<: *common
docker:
- image: pypy
environment:
TOXENV: pypy3-core
workflows:
version: 2
test:
jobs:
- docs
- lint
- py36-core
- py37-core
- pypy3-core

12
.circleci/merge_pr.sh Executable file
View File

@ -0,0 +1,12 @@
#!/usr/bin/env bash
if [[ -n "${CIRCLE_PR_NUMBER}" ]]; then
PR_INFO_URL=https://api.github.com/repos/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/pulls/$CIRCLE_PR_NUMBER
PR_BASE_BRANCH=$(curl -L "$PR_INFO_URL" | python -c 'import json, sys; obj = json.load(sys.stdin); sys.stdout.write(obj["base"]["ref"])')
git fetch origin +"$PR_BASE_BRANCH":circleci/pr-base
# We need these config values or git complains when creating the
# merge commit
git config --global user.name "Circle CI"
git config --global user.email "circleci@example.com"
git merge --no-edit circleci/pr-base
fi

38
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,38 @@
_If this is a bug report, please fill in the following sections.
If this is a feature request, delete and describe what you would like with examples._
## What was wrong?
### Code that produced the error
```py
CODE_TO_REPRODUCE
```
### Full error output
```sh
ERROR_HERE
```
### Expected Result
_This section may be deleted if the expectation is "don't crash"._
```sh
EXPECTED_RESULT
```
### Environment
```sh
# run this:
$ python -m eth_utils
# then copy the output here:
OUTPUT_HERE
```
## How can it be fixed?
Fill this section in if you know how this could or should be fixed.

21
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,21 @@
## What was wrong?
Issue #
## How was it fixed?
Summary of approach.
### To-Do
[//]: # (Stay ahead of things, add list items here!)
- [ ] Clean up commit history
[//]: # (For important changes that should go into the release notes please add a newsfragment file as explained here: https://github.com/libp2p/py-libp2p/blob/master/newsfragments/README.md)
[//]: # (See: https://py-libp2p.readthedocs.io/en/latest/contributing.html#pull-requests)
- [ ] Add entry to the [release notes](https://github.com/libp2p/py-libp2p/blob/master/newsfragments/README.md)
#### Cute Animal Picture
![put a cute animal picture link inside the parentheses]()

143
.gitignore vendored
View File

@ -1,56 +1,133 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
__pycache__/
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
*.egg-info
dist
build
eggs
.eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
venv*
.Python
downloads/
wheels/
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
pip-wheel-metadata
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
.tox
nosetests.xml
htmlcov/
.coverage.*
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
# Complexity
output/*.html
output/*/index.html
# Sphinx
docs/_build
docs/modules.rst
docs/*.internal.rst
docs/*._utils.*
# Hypothese Property base testing
.hypothesis
# tox/pytest cache
.cache
# Test output logs
logs
### JetBrains template
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
# User-specific stuff:
.idea/workspace.xml
.idea/tasks.xml
.idea/dictionaries
.idea/vcs.xml
.idea/jsLibraryMappings.xml
# Sensitive or high-churn files:
.idea/dataSources.ids
.idea/dataSources.xml
.idea/dataSources.local.xml
.idea/sqlDataSources.xml
.idea/dynamic.xml
.idea/uiDesigner.xml
# Gradle:
.idea/gradle.xml
.idea/libraries
# Mongo Explorer plugin:
.idea/mongoSettings.xml
# VIM temp files
*.sw[op]
# mypy
.mypy_cache
## File-based project format:
*.iws
## Plugin-specific files:
# IntelliJ
/out/
# mpeltonen/sbt-idea plugin
.idea_modules/
# JIRA plugin
atlassian-ide-plugin.xml
# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Django stuff:
*.log
local_settings.py
@ -63,9 +140,6 @@ instance/
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
@ -85,10 +159,8 @@ celerybeat-schedule
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
@ -100,8 +172,5 @@ venv.bak/
# mkdocs documentation
/site
# mypy
.mypy_cache/
# pycharm
.idea/
# vscode
.vscode/

View File

@ -0,0 +1,48 @@
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
PROJECT_ROOT=$(dirname $(dirname $(python -c 'import os, sys; sys.stdout.write(os.path.realpath(sys.argv[1]))' "$0")))
echo "What is your python module name?"
read MODULE_NAME
echo "What is your pypi package name? (default: $MODULE_NAME)"
read PYPI_INPUT
PYPI_NAME=${PYPI_INPUT:-$MODULE_NAME}
echo "What is your github project name? (default: $PYPI_NAME)"
read REPO_INPUT
REPO_NAME=${REPO_INPUT:-$PYPI_NAME}
echo "What is your readthedocs.org project name? (default: $PYPI_NAME)"
read RTD_INPUT
RTD_NAME=${RTD_INPUT:-$PYPI_NAME}
echo "What is your project name (ex: at the top of the README)? (default: $REPO_NAME)"
read PROJECT_INPUT
PROJECT_NAME=${PROJECT_INPUT:-$REPO_NAME}
echo "What is a one-liner describing the project?"
read SHORT_DESCRIPTION
_replace() {
local find_cmd=(find "$PROJECT_ROOT" ! -perm -u=x ! -path '*/.git/*' -type f)
if [[ $(uname) == Darwin ]]; then
"${find_cmd[@]}" -exec sed -i '' "$1" {} +
else
"${find_cmd[@]}" -exec sed -i "$1" {} +
fi
}
_replace "s/<MODULE_NAME>/$MODULE_NAME/g"
_replace "s/<PYPI_NAME>/$PYPI_NAME/g"
_replace "s/<REPO_NAME>/$REPO_NAME/g"
_replace "s/<RTD_NAME>/$RTD_NAME/g"
_replace "s/<PROJECT_NAME>/$PROJECT_NAME/g"
_replace "s/<SHORT_DESCRIPTION>/$SHORT_DESCRIPTION/g"
mkdir -p "$PROJECT_ROOT/$MODULE_NAME"
touch "$PROJECT_ROOT/$MODULE_NAME/__init__.py"

View File

@ -0,0 +1,2 @@
TEMPLATE_DIR=$(dirname $(readlink -f "$0"))
<"$TEMPLATE_DIR/template_vars.txt" "$TEMPLATE_DIR/fill_template_vars.sh"

View File

@ -0,0 +1,6 @@
libp2p
libp2p
py-libp2p
py-libp2p
py-libp2p
The Python implementation of the libp2p networking stack

30
.pydocstyle.ini Normal file
View File

@ -0,0 +1,30 @@
[pydocstyle]
; All error codes found here:
; http://www.pydocstyle.org/en/3.0.0/error_codes.html
;
; Ignored:
; D1 - Missing docstring error codes
;
; Selected:
; D2 - Whitespace error codes
; D3 - Quote error codes
; D4 - Content related error codes
select=D2,D3,D4
; Extra ignores:
; D200 - One-line docstring should fit on one line with quotes
; D203 - 1 blank line required before class docstring
; D204 - 1 blank line required after class docstring
; D205 - 1 blank line required between summary line and description
; D212 - Multi-line docstring summary should start at the first line
; D302 - Use u""" for Unicode docstrings
; D400 - First line should end with a period
; D401 - First line should be in imperative mood
; D412 - No blank lines allowed between a section header and its content
add-ignore=D200,D203,D204,D205,D212,D302,D400,D401,D412
; Explanation:
; D400 - Enabling this error code seems to make it a requirement that the first
; sentence in a docstring is not split across two lines. It also makes it a
; requirement that no docstring can have a multi-sentence description without a
; summary line. Neither one of those requirements seem appropriate.

533
.pylintrc
View File

@ -1,533 +0,0 @@
[MASTER]
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS
# Add files or directories matching the regex patterns to the blacklist. The
# regex matches against base names, not paths.
ignore-patterns=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Use multiple processes to speed up Pylint.
jobs=1
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
# Pickle collected data for later comparisons.
persistent=yes
# Specify a configuration file.
rcfile=.pylintrc
# When enabled, pylint would attempt to guess common misconfiguration and emit
# user-friendly hints instead of false-positive error messages
suggestion-mode=yes
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=print-statement,
parameter-unpacking,
unpacking-in-except,
old-raise-syntax,
backtick,
long-suffix,
old-ne-operator,
old-octal-literal,
import-star-module-level,
non-ascii-bytes-literal,
raw-checker-failed,
bad-inline-option,
locally-disabled,
locally-enabled,
file-ignored,
suppressed-message,
useless-suppression,
deprecated-pragma,
apply-builtin,
basestring-builtin,
buffer-builtin,
cmp-builtin,
coerce-builtin,
execfile-builtin,
file-builtin,
long-builtin,
raw_input-builtin,
reduce-builtin,
standarderror-builtin,
unicode-builtin,
xrange-builtin,
coerce-method,
delslice-method,
getslice-method,
setslice-method,
no-absolute-import,
old-division,
dict-iter-method,
dict-view-method,
next-method-called,
metaclass-assignment,
indexing-exception,
raising-string,
reload-builtin,
oct-method,
hex-method,
nonzero-method,
cmp-method,
input-builtin,
round-builtin,
intern-builtin,
unichr-builtin,
map-builtin-not-iterating,
zip-builtin-not-iterating,
range-builtin-not-iterating,
filter-builtin-not-iterating,
using-cmp-argument,
eq-without-hash,
div-method,
idiv-method,
rdiv-method,
exception-message-attribute,
invalid-str-codec,
sys-max-int,
bad-python3-import,
deprecated-string-function,
deprecated-str-translate-call,
deprecated-itertools-function,
deprecated-types-field,
next-method-defined,
dict-items-not-iterating,
dict-keys-not-iterating,
dict-values-not-iterating,
missing-docstring,
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=c-extension-no-member
[REPORTS]
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
# Set the output format. Available formats are text, parseable, colorized, json
# and msvs (visual studio).You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
output-format=text
# Tells whether to display a full report or only the messages
reports=no
# Activate the evaluation score.
score=no
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[SPELLING]
# Limits count of emitted suggestions for spelling mistakes
max-spelling-suggestions=4
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,
XXX
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The minimum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,
_cb
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=six.moves,past.builtins,future.builtins
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=" "
# Maximum number of characters on a single line.
max-line-length=100
# Maximum number of lines in a module
max-module-lines=1000
# List of optional constructs for which whitespace checking is disabled. `dict-
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
# `empty-line` allows space-only lines.
no-space-check=trailing-comma,
dict-separator
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[SIMILARITIES]
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=yes
# Minimum lines number of a similarity.
min-similarity-lines=8
[BASIC]
# Naming style matching correct argument names
argument-naming-style=snake_case
# Regular expression matching correct argument names. Overrides argument-
# naming-style
#argument-rgx=
# Naming style matching correct attribute names
attr-naming-style=snake_case
# Regular expression matching correct attribute names. Overrides attr-naming-
# style
#attr-rgx=
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,
bar,
baz,
toto,
tutu,
tata
# Naming style matching correct class attribute names
class-attribute-naming-style=any
# Regular expression matching correct class attribute names. Overrides class-
# attribute-naming-style
#class-attribute-rgx=
# Naming style matching correct class names
class-naming-style=PascalCase
# Regular expression matching correct class names. Overrides class-naming-style
#class-rgx=
# Naming style matching correct constant names
const-naming-style=UPPER_CASE
# Regular expression matching correct constant names. Overrides const-naming-
# style
#const-rgx=
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming style matching correct function names
function-naming-style=snake_case
# Regular expression matching correct function names. Overrides function-
# naming-style
#function-rgx=
# Good variable names which should always be accepted, separated by a comma
good-names=i,
j,
k,
ex,
Run,
_
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Naming style matching correct inline iteration names
inlinevar-naming-style=any
# Regular expression matching correct inline iteration names. Overrides
# inlinevar-naming-style
#inlinevar-rgx=
# Naming style matching correct method names
method-naming-style=snake_case
# Regular expression matching correct method names. Overrides method-naming-
# style
#method-rgx=
# Naming style matching correct module names
module-naming-style=snake_case
# Regular expression matching correct module names. Overrides module-naming-
# style
#module-rgx=
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
property-classes=abc.abstractproperty
# Naming style matching correct variable names
variable-naming-style=snake_case
# Regular expression matching correct variable names. Overrides variable-
# naming-style
#variable-rgx=
[IMPORTS]
# Allow wildcard imports from modules that define __all__.
allow-wildcard-with-all=no
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=optparse,tkinter.tix
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
# Force import order to recognize a module as part of the standard
# compatibility libraries.
known-standard-library=
# Force import order to recognize a module as part of a third party library.
known-third-party=enchant
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,
__new__,
setUp
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,
_fields,
_replace,
_source,
_make
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# Maximum number of arguments for function / method
max-args=5
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of statements in function / method body
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=Exception

View File

@ -2,22 +2,36 @@ language: python
matrix:
include:
- python: 3.6-dev
dist: xenial
env: TOXENV=py36-test
- python: 3.7
dist: xenial
env: TOXENV=py37-test
- python: 3.7
dist: xenial
env: TOXENV=lint
- python: 3.7
dist: xenial
env: TOXENV=docs
- python: 3.7
dist: xenial
env: TOXENV=py37-interop GOBINPKG=go1.13.8.linux-amd64.tar.gz
sudo: true
before_install:
- wget https://dl.google.com/go/$GOBINPKG
- sudo tar -C /usr/local -xzf $GOBINPKG
- export GOPATH=$HOME/go
- export GOROOT=/usr/local/go
- export PATH=$GOROOT/bin:$GOPATH/bin:$PATH
- ./tests_interop/go_pkgs/install_interop_go_pkgs.sh
install:
- pip install --upgrade pip
- pip install -r requirements_dev.txt
- python setup.py develop
- pip install tox
script:
- pytest --cov=./libp2p tests/
- pylint --rcfile=.pylintrc libp2p/!(kademlia) tests
after_success:
- codecov
- tox
notifications:
slack: py-libp2p:RK0WVoQZhQXLgIKfHNPL1TR2

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2019 The Ethereum Foundation
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

113
Makefile Normal file
View File

@ -0,0 +1,113 @@
CURRENT_SIGN_SETTING := $(shell git config commit.gpgSign)
.PHONY: clean-pyc clean-build docs
help:
@echo "clean-build - remove build artifacts"
@echo "clean-pyc - remove Python file artifacts"
@echo "lint - check style with flake8, etc"
@echo "lint-roll - auto-correct styles with isort, black, docformatter, etc"
@echo "test - run tests quickly with the default Python"
@echo "testall - run tests on every Python version with tox"
@echo "release - package and upload a release"
@echo "dist - package"
FILES_TO_LINT = libp2p tests tests_interop examples setup.py
PB = libp2p/crypto/pb/crypto.proto \
libp2p/pubsub/pb/rpc.proto \
libp2p/security/insecure/pb/plaintext.proto \
libp2p/security/secio/pb/spipe.proto \
libp2p/security/noise/pb/noise.proto \
libp2p/identity/identify/pb/identify.proto
PY = $(PB:.proto=_pb2.py)
PYI = $(PB:.proto=_pb2.pyi)
# Set default to `protobufs`, otherwise `format` is called when typing only `make`
all: protobufs
protobufs: $(PY)
%_pb2.py: %.proto
protoc --python_out=. --mypy_out=. $<
clean-proto:
rm -f $(PY) $(PYI)
clean: clean-build clean-pyc
clean-build:
rm -fr build/
rm -fr dist/
rm -fr *.egg-info
clean-pyc:
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
find . -name '__pycache__' -exec rm -rf {} +
lint:
mypy -p libp2p -p examples --config-file mypy.ini
flake8 $(FILES_TO_LINT)
black --check $(FILES_TO_LINT)
isort --recursive --check-only --diff $(FILES_TO_LINT)
docformatter --pre-summary-newline --check --recursive $(FILES_TO_LINT)
tox -e lint # This is probably redundant, but just in case...
lint-roll:
isort --recursive $(FILES_TO_LINT)
black $(FILES_TO_LINT)
docformatter -ir --pre-summary-newline $(FILES_TO_LINT)
$(MAKE) lint
test:
pytest tests
test-all:
tox
build-docs:
sphinx-apidoc -o docs/ . setup.py "*conftest*" "libp2p/tools/interop*"
$(MAKE) -C docs clean
$(MAKE) -C docs html
$(MAKE) -C docs doctest
./newsfragments/validate_files.py
towncrier --draft --version preview
docs: build-docs
open docs/_build/html/index.html
linux-docs: build-docs
xdg-open docs/_build/html/index.html
package: clean
python setup.py sdist bdist_wheel
python scripts/release/test_package.py
notes:
# Let UPCOMING_VERSION be the version that is used for the current bump
$(eval UPCOMING_VERSION=$(shell bumpversion $(bump) --dry-run --list | grep new_version= | sed 's/new_version=//g'))
# Now generate the release notes to have them included in the release commit
towncrier --yes --version $(UPCOMING_VERSION)
# Before we bump the version, make sure that the towncrier-generated docs will build
make build-docs
git commit -m "Compile release notes"
release: clean
# require that you be on a branch that's linked to upstream/master
git status -s -b | head -1 | grep "\.\.upstream/master"
# verify that docs build correctly
./newsfragments/validate_files.py is-empty
make build-docs
CURRENT_SIGN_SETTING=$(git config commit.gpgSign)
git config commit.gpgSign true
bumpversion $(bump)
git push upstream && git push upstream --tags
python setup.py sdist bdist_wheel
twine upload dist/*
git config commit.gpgSign "$(CURRENT_SIGN_SETTING)"
dist: clean
python setup.py sdist bdist_wheel
ls -l dist

103
README.md
View File

@ -1,39 +1,86 @@
# py-libp2p [![Build Status](https://travis-ci.com/zixuanzh/py-libp2p.svg?branch=master)](https://travis-ci.com/zixuanzh/py-libp2p) [![codecov](https://codecov.io/gh/zixuanzh/py-libp2p/branch/master/graph/badge.svg)](https://codecov.io/gh/zixuanzh/py-libp2p) [![Gitter chat](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/py-libp2p/Lobby)[![Freenode](https://img.shields.io/badge/freenode-%23libp2p-yellow.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23libp2p)
# py-libp2p
[![Join the chat at https://gitter.im/py-libp2p/Lobby](https://badges.gitter.im/py-libp2p/Lobby.png)](https://gitter.im/py-libp2p/Lobby)
[![Build Status](https://travis-ci.com/libp2p/py-libp2p.svg?branch=master)](https://travis-ci.com/libp2p/py-libp2p)
[![PyPI version](https://badge.fury.io/py/libp2p.svg)](https://badge.fury.io/py/libp2p)
[![Python versions](https://img.shields.io/pypi/pyversions/libp2p.svg)](https://pypi.python.org/pypi/libp2p)
[![Docs build](https://readthedocs.org/projects/py-libp2p/badge/?version=latest)](http://py-libp2p.readthedocs.io/en/latest/?badge=latest)
[![Freenode](https://img.shields.io/badge/freenode-%23libp2p-yellow.svg)](https://webchat.freenode.net/?channels=%23libp2p)
[![Matrix](https://img.shields.io/badge/matrix-%23libp2p%3Apermaweb.io-blue.svg)](https://riot.permaweb.io/#/room/#libp2p:permaweb.io)
[![Discord](https://img.shields.io/discord/475789330380488707?color=blueviolet&label=discord)](https://discord.gg/66KBrm2)
<h1 align="center">
<img width="250" align="center" src="https://github.com/zixuanzh/py-libp2p/blob/master/assets/py-libp2p-logo.png?raw=true" alt="py-libp2p hex logo" />
<img width="250" align="center" src="https://github.com/libp2p/py-libp2p/blob/master/assets/py-libp2p-logo.png?raw=true" alt="py-libp2p hex logo" />
</h1>
## WARNING
py-libp2p is an experimental and work-in-progress repo under heavy development. We do not yet recommend using py-libp2p in production environments.
The Python implementation of the libp2p networking stack
Read more in the [documentation on ReadTheDocs](https://py-libp2p.readthedocs.io/). [View the release notes](https://py-libp2p.readthedocs.io/en/latest/release_notes.html).
## Sponsorship
This project is graciously sponsored by the Ethereum Foundation through [Wave 5 of their Grants Program](https://blog.ethereum.org/2019/02/21/ethereum-foundation-grants-program-wave-5/).
## Maintainers
The py-libp2p team consists of:
[@zixuanzh](https://github.com/zixuanzh) [@alexh](https://github.com/alexh) [@stuckinaboot](https://github.com/stuckinaboot) [@robzajac](https://github.com/robzajac)
[@zixuanzh](https://github.com/zixuanzh) [@alexh](https://github.com/alexh) [@stuckinaboot](https://github.com/stuckinaboot) [@robzajac](https://github.com/robzajac) [@carver](https://github.com/carver)
## Development
py-libp2p requires Python 3.7 and the best way to guarantee a clean Python 3.7 environment is with [`virtualenv`](https://virtualenv.pypa.io/en/stable/)
```sh
git clone git@github.com:libp2p/py-libp2p.git
cd py-libp2p
virtualenv -p python3.7 venv
. venv/bin/activate
pip3 install -r requirements_dev.txt
python setup.py develop
pip install -e .[dev]
```
## Testing
### Testing Setup
During development, you might like to have tests run on every file save.
Show flake8 errors on file change:
After installing our requirements (see above), you can:
```sh
cd tests
pytest
# Test flake8
when-changed -v -s -r -1 libp2p/ tests/ -c "clear; flake8 libp2p tests && echo 'flake8 success' || echo 'error'"
```
Run multi-process tests in one command, but without color:
```sh
# in the project root:
pytest --numprocesses=4 --looponfail --maxfail=1
# the same thing, succinctly:
pytest -n 4 -f --maxfail=1
```
Run in one thread, with color and desktop notifications:
```sh
cd venv
ptw --onfail "notify-send -t 5000 'Test failure ⚠⚠⚠⚠⚠' 'python 3 test on py-libp2p failed'" ../tests ../libp2p
```
Note that tests/libp2p/test_libp2p.py contains an end-to-end messaging test between two libp2p hosts, which is the bulk of our proof of concept.
### Release setup
Releases follow the same basic pattern as releases of some tangentially-related projects,
like Trinity. See [Trinity's release instructions](
https://trinity-client.readthedocs.io/en/latest/contributing.html#releasing).
## Requirements
The protobuf description in this repository was generated by `protoc` at version `3.7.1`.
## Feature Breakdown
py-libp2p aims for conformity with [the standard libp2p modules](https://github.com/libp2p/libp2p/blob/master/REQUIREMENTS.md#libp2p-modules-implementations). Below is a breakdown of the modules we have developed, are developing, and may develop in the future.
@ -46,16 +93,16 @@ py-libp2p aims for conformity with [the standard libp2p modules](https://github.
| Identify Protocol | Status |
| -------------------------------------------- | :-----------: |
| **`Identify`** | :tomato: |
| **`Identify`** | :lemon: |
| Transport Protocols | Status |
| -------------------------------------------- | :-----------: |
| **`TCP`** | :lemon: tests |
| **`TCP`** | :green_apple: |
| **`UDP`** | :tomato: |
| **`WebSockets`** | :tomato: |
| **`UTP`** | :tomato: |
| **`WebRTC`** | :tomato: |
| **`WebSockets`** | :chestnut: |
| **`UTP`** | :chestnut: |
| **`WebRTC`** | :chestnut: |
| **`SCTP`** | :chestnut: |
| **`Tor`** | :chestnut: |
| **`i2p`** | :chestnut: |
@ -68,7 +115,7 @@ py-libp2p aims for conformity with [the standard libp2p modules](https://github.
| Stream Muxers | Status |
| -------------------------------------------- | :-----------: |
| **`multiplex`** | :lemon: tests |
| **`multiplex`** | :green_apple: |
| **`yamux`** | :tomato: |
| **`benchmarks`** | :chestnut: |
| **`muxado`** | :chestnut: |
@ -85,39 +132,39 @@ py-libp2p aims for conformity with [the standard libp2p modules](https://github.
| Switch (Swarm) | Status |
| -------------------------------------------- | :-----------: |
| **`Switch`** | :lemon: tests |
| **`Dialer stack`** | :chestnut: |
| **`Switch`** | :green_apple: |
| **`Dialer stack`** | :green_apple: |
| Peer Discovery | Status |
| -------------------------------------------- | :-----------: |
| **`bootstrap list`** | :green_apple: |
| **`Kademlia DHT`** | :tomato: |
| **`mDNS`** | :tomato: |
| **`bootstrap list`** | :tomato: |
| **`Kademlia DHT`** | :chestnut: |
| **`mDNS`** | :chestnut: |
| **`PEX`** | :chestnut: |
| **`DNS`** | :chestnut: |
| Content Routing | Status |
| -------------------------------------------- | :-----------: |
| **`Kademlia DHT`** | :tomato: |
| **`floodsub`** | :tomato: |
| **`gossipsub`** | :tomato: |
| **`Kademlia DHT`** | :chestnut: |
| **`floodsub`** | :green_apple: |
| **`gossipsub`** | :green_apple: |
| **`PHT`** | :chestnut: |
| Peer Routing | Status |
| -------------------------------------------- | :-----------: |
| **`Kademlia DHT`** | :tomato: |
| **`floodsub`** | :tomato: |
| **`gossipsub`** | :tomato: |
| **`Kademlia DHT`** | :chestnut: |
| **`floodsub`** | :green_apple: |
| **`gossipsub`** | :green_apple: |
| **`PHT`** | :chestnut: |
| NAT Traversal | Status |
| -------------------------------------------- | :-----------: |
| **`nat-pmp`** | :tomato: |
| **`upnp`** | :tomato: |
| **`nat-pmp`** | :chestnut: |
| **`upnp`** | :chestnut: |
| **`ext addr discovery`** | :chestnut: |
| **`STUN-like`** | :chestnut: |
| **`line-switch relay`** | :chestnut: |

177
docs/Makefile Normal file
View File

@ -0,0 +1,177 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS = -W
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/web3.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/web3.qhc"
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/web3"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/web3"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

304
docs/conf.py Normal file
View File

@ -0,0 +1,304 @@
# -*- coding: utf-8 -*-
#
# py-libp2p documentation build configuration file, created by
# sphinx-quickstart on Thu Oct 16 20:43:24 2014.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
import os
DIR = os.path.dirname('__file__')
with open (os.path.join(DIR, '../setup.py'), 'r') as f:
for line in f:
if 'version=' in line:
setup_version = line.split('"')[1]
break
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.doctest',
'sphinx.ext.intersphinx',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = 'py-libp2p'
copyright = '2019, The Ethereum Foundation'
__version__ = setup_version
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '.'.join(__version__.split('.')[:2])
# The full version, including alpha/beta/rc tags.
release = __version__
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = [
'_build',
'modules.rst',
]
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'sphinx_rtd_theme'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'libp2pdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
('index', 'libp2p.tex', 'py-libp2p Documentation',
'The Ethereum Foundation', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'libp2p', 'py-libp2p Documentation',
['The Ethereum Foundation'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'py-libp2p', 'py-libp2p Documentation',
'The Ethereum Foundation', 'py-libp2p', 'The Python implementation of the libp2p networking stack',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
# -- Intersphinx configuration ------------------------------------------------
intersphinx_mapping = {
'python': ('https://docs.python.org/3.6', None),
}
# -- Doctest configuration ----------------------------------------
import doctest
doctest_default_flags = (0
| doctest.DONT_ACCEPT_TRUE_FOR_1
| doctest.ELLIPSIS
| doctest.IGNORE_EXCEPTION_DETAIL
| doctest.NORMALIZE_WHITESPACE
)
# -- Mocked dependencies ----------------------------------------
# Mock out dependencies that are unbuildable on readthedocs, as recommended here:
# https://docs.readthedocs.io/en/rel/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
import sys
from unittest.mock import MagicMock
# Add new modules to mock here (it should be the same list as those excluded in setup.py)
MOCK_MODULES = [
"fastecdsa",
"fastecdsa.encoding",
"fastecdsa.encoding.sec1",
]
sys.modules.update((mod_name, MagicMock()) for mod_name in MOCK_MODULES)

22
docs/examples.chat.rst Normal file
View File

@ -0,0 +1,22 @@
examples.chat package
=====================
Submodules
----------
examples.chat.chat module
-------------------------
.. automodule:: examples.chat.chat
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: examples.chat
:members:
:undoc-members:
:show-inheritance:

17
docs/examples.rst Normal file
View File

@ -0,0 +1,17 @@
examples package
================
Subpackages
-----------
.. toctree::
examples.chat
Module contents
---------------
.. automodule:: examples
:members:
:undoc-members:
:show-inheritance:

21
docs/index.rst Normal file
View File

@ -0,0 +1,21 @@
py-libp2p
==============================
The Python implementation of the libp2p networking stack
Contents
--------
.. toctree::
:maxdepth: 3
libp2p
release_notes
examples
Indices and tables
------------------
* :ref:`genindex`
* :ref:`modindex`

22
docs/libp2p.crypto.pb.rst Normal file
View File

@ -0,0 +1,22 @@
libp2p.crypto.pb package
========================
Submodules
----------
libp2p.crypto.pb.crypto\_pb2 module
-----------------------------------
.. automodule:: libp2p.crypto.pb.crypto_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.crypto.pb
:members:
:undoc-members:
:show-inheritance:

93
docs/libp2p.crypto.rst Normal file
View File

@ -0,0 +1,93 @@
libp2p.crypto package
=====================
Subpackages
-----------
.. toctree::
libp2p.crypto.pb
Submodules
----------
libp2p.crypto.authenticated\_encryption module
----------------------------------------------
.. automodule:: libp2p.crypto.authenticated_encryption
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.ecc module
------------------------
.. automodule:: libp2p.crypto.ecc
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.ed25519 module
----------------------------
.. automodule:: libp2p.crypto.ed25519
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.exceptions module
-------------------------------
.. automodule:: libp2p.crypto.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.key\_exchange module
----------------------------------
.. automodule:: libp2p.crypto.key_exchange
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.keys module
-------------------------
.. automodule:: libp2p.crypto.keys
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.rsa module
------------------------
.. automodule:: libp2p.crypto.rsa
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.secp256k1 module
------------------------------
.. automodule:: libp2p.crypto.secp256k1
:members:
:undoc-members:
:show-inheritance:
libp2p.crypto.serialization module
----------------------------------
.. automodule:: libp2p.crypto.serialization
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.crypto
:members:
:undoc-members:
:show-inheritance:

62
docs/libp2p.host.rst Normal file
View File

@ -0,0 +1,62 @@
libp2p.host package
===================
Submodules
----------
libp2p.host.basic\_host module
------------------------------
.. automodule:: libp2p.host.basic_host
:members:
:undoc-members:
:show-inheritance:
libp2p.host.defaults module
---------------------------
.. automodule:: libp2p.host.defaults
:members:
:undoc-members:
:show-inheritance:
libp2p.host.exceptions module
-----------------------------
.. automodule:: libp2p.host.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.host.host\_interface module
----------------------------------
.. automodule:: libp2p.host.host_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.host.ping module
-----------------------
.. automodule:: libp2p.host.ping
:members:
:undoc-members:
:show-inheritance:
libp2p.host.routed\_host module
-------------------------------
.. automodule:: libp2p.host.routed_host
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.host
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.identity.identify.pb package
===================================
Submodules
----------
libp2p.identity.identify.pb.identify\_pb2 module
------------------------------------------------
.. automodule:: libp2p.identity.identify.pb.identify_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.identity.identify.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,29 @@
libp2p.identity.identify package
================================
Subpackages
-----------
.. toctree::
libp2p.identity.identify.pb
Submodules
----------
libp2p.identity.identify.protocol module
----------------------------------------
.. automodule:: libp2p.identity.identify.protocol
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.identity.identify
:members:
:undoc-members:
:show-inheritance:

17
docs/libp2p.identity.rst Normal file
View File

@ -0,0 +1,17 @@
libp2p.identity package
=======================
Subpackages
-----------
.. toctree::
libp2p.identity.identify
Module contents
---------------
.. automodule:: libp2p.identity
:members:
:undoc-members:
:show-inheritance:

46
docs/libp2p.io.rst Normal file
View File

@ -0,0 +1,46 @@
libp2p.io package
=================
Submodules
----------
libp2p.io.abc module
--------------------
.. automodule:: libp2p.io.abc
:members:
:undoc-members:
:show-inheritance:
libp2p.io.exceptions module
---------------------------
.. automodule:: libp2p.io.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.io.msgio module
----------------------
.. automodule:: libp2p.io.msgio
:members:
:undoc-members:
:show-inheritance:
libp2p.io.utils module
----------------------
.. automodule:: libp2p.io.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.io
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,54 @@
libp2p.network.connection package
=================================
Submodules
----------
libp2p.network.connection.exceptions module
-------------------------------------------
.. automodule:: libp2p.network.connection.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.net\_connection\_interface module
-----------------------------------------------------------
.. automodule:: libp2p.network.connection.net_connection_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.raw\_connection module
------------------------------------------------
.. automodule:: libp2p.network.connection.raw_connection
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.raw\_connection\_interface module
-----------------------------------------------------------
.. automodule:: libp2p.network.connection.raw_connection_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.connection.swarm\_connection module
--------------------------------------------------
.. automodule:: libp2p.network.connection.swarm_connection
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.network.connection
:members:
:undoc-members:
:show-inheritance:

54
docs/libp2p.network.rst Normal file
View File

@ -0,0 +1,54 @@
libp2p.network package
======================
Subpackages
-----------
.. toctree::
libp2p.network.connection
libp2p.network.stream
Submodules
----------
libp2p.network.exceptions module
--------------------------------
.. automodule:: libp2p.network.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.network.network\_interface module
----------------------------------------
.. automodule:: libp2p.network.network_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.notifee\_interface module
----------------------------------------
.. automodule:: libp2p.network.notifee_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.network.swarm module
---------------------------
.. automodule:: libp2p.network.swarm
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.network
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,38 @@
libp2p.network.stream package
=============================
Submodules
----------
libp2p.network.stream.exceptions module
---------------------------------------
.. automodule:: libp2p.network.stream.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.network.stream.net\_stream module
----------------------------------------
.. automodule:: libp2p.network.stream.net_stream
:members:
:undoc-members:
:show-inheritance:
libp2p.network.stream.net\_stream\_interface module
---------------------------------------------------
.. automodule:: libp2p.network.stream.net_stream_interface
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.network.stream
:members:
:undoc-members:
:show-inheritance:

78
docs/libp2p.peer.rst Normal file
View File

@ -0,0 +1,78 @@
libp2p.peer package
===================
Submodules
----------
libp2p.peer.addrbook\_interface module
--------------------------------------
.. automodule:: libp2p.peer.addrbook_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.id module
---------------------
.. automodule:: libp2p.peer.id
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerdata module
---------------------------
.. automodule:: libp2p.peer.peerdata
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerdata\_interface module
--------------------------------------
.. automodule:: libp2p.peer.peerdata_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerinfo module
---------------------------
.. automodule:: libp2p.peer.peerinfo
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peermetadata\_interface module
------------------------------------------
.. automodule:: libp2p.peer.peermetadata_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerstore module
----------------------------
.. automodule:: libp2p.peer.peerstore
:members:
:undoc-members:
:show-inheritance:
libp2p.peer.peerstore\_interface module
---------------------------------------
.. automodule:: libp2p.peer.peerstore_interface
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.peer
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,70 @@
libp2p.protocol\_muxer package
==============================
Submodules
----------
libp2p.protocol\_muxer.exceptions module
----------------------------------------
.. automodule:: libp2p.protocol_muxer.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect module
-----------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_client module
-------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_client
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_client\_interface module
------------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_client_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_communicator module
-------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_communicator
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_communicator\_interface module
------------------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_communicator_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.protocol\_muxer.multiselect\_muxer\_interface module
-----------------------------------------------------------
.. automodule:: libp2p.protocol_muxer.multiselect_muxer_interface
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.protocol_muxer
:members:
:undoc-members:
:show-inheritance:

22
docs/libp2p.pubsub.pb.rst Normal file
View File

@ -0,0 +1,22 @@
libp2p.pubsub.pb package
========================
Submodules
----------
libp2p.pubsub.pb.rpc\_pb2 module
--------------------------------
.. automodule:: libp2p.pubsub.pb.rpc_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.pubsub.pb
:members:
:undoc-members:
:show-inheritance:

93
docs/libp2p.pubsub.rst Normal file
View File

@ -0,0 +1,93 @@
libp2p.pubsub package
=====================
Subpackages
-----------
.. toctree::
libp2p.pubsub.pb
Submodules
----------
libp2p.pubsub.abc module
------------------------
.. automodule:: libp2p.pubsub.abc
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.exceptions module
-------------------------------
.. automodule:: libp2p.pubsub.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.floodsub module
-----------------------------
.. automodule:: libp2p.pubsub.floodsub
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.gossipsub module
------------------------------
.. automodule:: libp2p.pubsub.gossipsub
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.mcache module
---------------------------
.. automodule:: libp2p.pubsub.mcache
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.pubsub module
---------------------------
.. automodule:: libp2p.pubsub.pubsub
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.pubsub\_notifee module
------------------------------------
.. automodule:: libp2p.pubsub.pubsub_notifee
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.subscription module
---------------------------------
.. automodule:: libp2p.pubsub.subscription
:members:
:undoc-members:
:show-inheritance:
libp2p.pubsub.validators module
-------------------------------
.. automodule:: libp2p.pubsub.validators
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.pubsub
:members:
:undoc-members:
:show-inheritance:

23
docs/libp2p.routing.rst Normal file
View File

@ -0,0 +1,23 @@
libp2p.routing package
======================
Submodules
----------
libp2p.routing.interfaces module
--------------------------------
.. automodule:: libp2p.routing.interfaces
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.routing
:members:
:undoc-members:
:show-inheritance:

57
docs/libp2p.rst Normal file
View File

@ -0,0 +1,57 @@
libp2p package
==============
Subpackages
-----------
.. toctree::
libp2p.crypto
libp2p.host
libp2p.identity
libp2p.io
libp2p.network
libp2p.peer
libp2p.protocol_muxer
libp2p.pubsub
libp2p.routing
libp2p.security
libp2p.stream_muxer
libp2p.tools
libp2p.transport
Submodules
----------
libp2p.exceptions module
------------------------
.. automodule:: libp2p.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.typing module
--------------------
.. automodule:: libp2p.typing
:members:
:undoc-members:
:show-inheritance:
libp2p.utils module
-------------------
.. automodule:: libp2p.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.security.insecure.pb package
===================================
Submodules
----------
libp2p.security.insecure.pb.plaintext\_pb2 module
-------------------------------------------------
.. automodule:: libp2p.security.insecure.pb.plaintext_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.insecure.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,29 @@
libp2p.security.insecure package
================================
Subpackages
-----------
.. toctree::
libp2p.security.insecure.pb
Submodules
----------
libp2p.security.insecure.transport module
-----------------------------------------
.. automodule:: libp2p.security.insecure.transport
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.insecure
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.security.noise.pb package
================================
Submodules
----------
libp2p.security.noise.pb.noise\_pb2 module
------------------------------------------
.. automodule:: libp2p.security.noise.pb.noise_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.noise.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,61 @@
libp2p.security.noise package
=============================
Subpackages
-----------
.. toctree::
libp2p.security.noise.pb
Submodules
----------
libp2p.security.noise.exceptions module
---------------------------------------
.. automodule:: libp2p.security.noise.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.io module
-------------------------------
.. automodule:: libp2p.security.noise.io
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.messages module
-------------------------------------
.. automodule:: libp2p.security.noise.messages
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.patterns module
-------------------------------------
.. automodule:: libp2p.security.noise.patterns
:members:
:undoc-members:
:show-inheritance:
libp2p.security.noise.transport module
--------------------------------------
.. automodule:: libp2p.security.noise.transport
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.noise
:members:
:undoc-members:
:show-inheritance:

71
docs/libp2p.security.rst Normal file
View File

@ -0,0 +1,71 @@
libp2p.security package
=======================
Subpackages
-----------
.. toctree::
libp2p.security.insecure
libp2p.security.noise
libp2p.security.secio
Submodules
----------
libp2p.security.base\_session module
------------------------------------
.. automodule:: libp2p.security.base_session
:members:
:undoc-members:
:show-inheritance:
libp2p.security.base\_transport module
--------------------------------------
.. automodule:: libp2p.security.base_transport
:members:
:undoc-members:
:show-inheritance:
libp2p.security.exceptions module
---------------------------------
.. automodule:: libp2p.security.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.security.secure\_conn\_interface module
----------------------------------------------
.. automodule:: libp2p.security.secure_conn_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.security.secure\_transport\_interface module
---------------------------------------------------
.. automodule:: libp2p.security.secure_transport_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.security.security\_multistream module
--------------------------------------------
.. automodule:: libp2p.security.security_multistream
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.security.secio.pb package
================================
Submodules
----------
libp2p.security.secio.pb.spipe\_pb2 module
------------------------------------------
.. automodule:: libp2p.security.secio.pb.spipe_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.secio.pb
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,37 @@
libp2p.security.secio package
=============================
Subpackages
-----------
.. toctree::
libp2p.security.secio.pb
Submodules
----------
libp2p.security.secio.exceptions module
---------------------------------------
.. automodule:: libp2p.security.secio.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.security.secio.transport module
--------------------------------------
.. automodule:: libp2p.security.secio.transport
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.security.secio
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,54 @@
libp2p.stream\_muxer.mplex package
==================================
Submodules
----------
libp2p.stream\_muxer.mplex.constants module
-------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.constants
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.datastructures module
------------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.datastructures
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.exceptions module
--------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.mplex module
---------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.mplex
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.mplex.mplex\_stream module
-----------------------------------------------
.. automodule:: libp2p.stream_muxer.mplex.mplex_stream
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.stream_muxer.mplex
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,45 @@
libp2p.stream\_muxer package
============================
Subpackages
-----------
.. toctree::
libp2p.stream_muxer.mplex
Submodules
----------
libp2p.stream\_muxer.abc module
-------------------------------
.. automodule:: libp2p.stream_muxer.abc
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.exceptions module
--------------------------------------
.. automodule:: libp2p.stream_muxer.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.stream\_muxer.muxer\_multistream module
----------------------------------------------
.. automodule:: libp2p.stream_muxer.muxer_multistream
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.stream_muxer
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,38 @@
libp2p.tools.pubsub package
===========================
Submodules
----------
libp2p.tools.pubsub.dummy\_account\_node module
-----------------------------------------------
.. automodule:: libp2p.tools.pubsub.dummy_account_node
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.pubsub.floodsub\_integration\_test\_settings module
----------------------------------------------------------------
.. automodule:: libp2p.tools.pubsub.floodsub_integration_test_settings
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.pubsub.utils module
--------------------------------
.. automodule:: libp2p.tools.pubsub.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.tools.pubsub
:members:
:undoc-members:
:show-inheritance:

47
docs/libp2p.tools.rst Normal file
View File

@ -0,0 +1,47 @@
libp2p.tools package
====================
Subpackages
-----------
.. toctree::
libp2p.tools.pubsub
The interop module is left out for now, because of the extra dependencies it requires.
Submodules
----------
libp2p.tools.constants module
-----------------------------
.. automodule:: libp2p.tools.constants
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.factories module
-----------------------------
.. automodule:: libp2p.tools.factories
:members:
:undoc-members:
:show-inheritance:
libp2p.tools.utils module
-------------------------
.. automodule:: libp2p.tools.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.tools
:members:
:undoc-members:
:show-inheritance:

61
docs/libp2p.transport.rst Normal file
View File

@ -0,0 +1,61 @@
libp2p.transport package
========================
Subpackages
-----------
.. toctree::
libp2p.transport.tcp
Submodules
----------
libp2p.transport.exceptions module
----------------------------------
.. automodule:: libp2p.transport.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.listener\_interface module
-------------------------------------------
.. automodule:: libp2p.transport.listener_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.transport\_interface module
--------------------------------------------
.. automodule:: libp2p.transport.transport_interface
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.typing module
------------------------------
.. automodule:: libp2p.transport.typing
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.upgrader module
--------------------------------
.. automodule:: libp2p.transport.upgrader
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.transport
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.transport.tcp package
============================
Submodules
----------
libp2p.transport.tcp.tcp module
-------------------------------
.. automodule:: libp2p.transport.tcp.tcp
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.transport.tcp
:members:
:undoc-members:
:show-inheritance:

95
docs/release_notes.rst Normal file
View File

@ -0,0 +1,95 @@
Release Notes
=============
.. towncrier release notes start
libp2p v0.1.5 (2020-03-25)
---------------------------
Features
~~~~~~~~
- Dial all multiaddrs stored for a peer when attempting to connect (not just the first one in the peer store). (`#386 <https://github.com/libp2p/py-libp2p/issues/386>`__)
- Migrate transport stack to trio-compatible code. Merge in #404. (`#396 <https://github.com/libp2p/py-libp2p/issues/396>`__)
- Migrate network stack to trio-compatible code. Merge in #404. (`#397 <https://github.com/libp2p/py-libp2p/issues/397>`__)
- Migrate host, peer and protocols stacks to trio-compatible code. Merge in #404. (`#398 <https://github.com/libp2p/py-libp2p/issues/398>`__)
- Migrate muxer and security transport stacks to trio-compatible code. Merge in #404. (`#399 <https://github.com/libp2p/py-libp2p/issues/399>`__)
- Migrate pubsub stack to trio-compatible code. Merge in #404. (`#400 <https://github.com/libp2p/py-libp2p/issues/400>`__)
- Fix interop tests w/ new trio-style code. Merge in #404. (`#401 <https://github.com/libp2p/py-libp2p/issues/401>`__)
- Fix remainder of test code w/ new trio-style code. Merge in #404. (`#402 <https://github.com/libp2p/py-libp2p/issues/402>`__)
- Add initial infrastructure for `noise` security transport. (`#405 <https://github.com/libp2p/py-libp2p/issues/405>`__)
- Add `PatternXX` of `noise` security transport. (`#406 <https://github.com/libp2p/py-libp2p/issues/406>`__)
- The `msg_id` in a pubsub message is now configurable by the user of the library. (`#410 <https://github.com/libp2p/py-libp2p/issues/410>`__)
Bugfixes
~~~~~~~~
- Use `sha256` when calculating a peer's ID from their public key in Kademlia DHTs. (`#385 <https://github.com/libp2p/py-libp2p/issues/385>`__)
- Store peer ids in ``set`` instead of ``list`` and check if peer id exists in ``dict`` before accessing to prevent ``KeyError``. (`#387 <https://github.com/libp2p/py-libp2p/issues/387>`__)
- Do not close a connection if it has been reset. (`#394 <https://github.com/libp2p/py-libp2p/issues/394>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Add support for `fastecdsa` on windows (and thereby supporting windows installation via `pip`) (`#380 <https://github.com/libp2p/py-libp2p/issues/380>`__)
- Prefer f-string style formatting everywhere except logging statements. (`#389 <https://github.com/libp2p/py-libp2p/issues/389>`__)
- Mark `lru` dependency as third-party to fix a windows inconsistency. (`#392 <https://github.com/libp2p/py-libp2p/issues/392>`__)
- Bump `multiaddr` dependency to version `0.0.9` so that multiaddr objects are hashable. (`#393 <https://github.com/libp2p/py-libp2p/issues/393>`__)
- Remove incremental mode of mypy to disable some warnings. (`#403 <https://github.com/libp2p/py-libp2p/issues/403>`__)
libp2p v0.1.4 (2019-12-12)
--------------------------
Features
~~~~~~~~
- Added support for Python 3.6 (`#372 <https://github.com/libp2p/py-libp2p/issues/372>`__)
- Add signing and verification to pubsub (`#362 <https://github.com/libp2p/py-libp2p/issues/362>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Refactor and cleanup gossipsub (`#373 <https://github.com/libp2p/py-libp2p/issues/373>`__)
libp2p v0.1.3 (2019-11-27)
--------------------------
Bugfixes
~~~~~~~~
- Handle Stream* errors (like ``StreamClosed``) during calls to ``stream.write()`` and
``stream.read()`` (`#350 <https://github.com/libp2p/py-libp2p/issues/350>`__)
- Relax the protobuf dependency to play nicely with other libraries. It was pinned to 3.9.0, and now
permits v3.10 up to (but not including) v4. (`#354 <https://github.com/libp2p/py-libp2p/issues/354>`__)
- Fixes KeyError when peer in a stream accidentally closes and resets the stream, because handlers
for both will try to ``del streams[stream_id]`` without checking if the entry still exists. (`#355 <https://github.com/libp2p/py-libp2p/issues/355>`__)
Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~
- Use Sphinx & autodoc to generate docs, now available on `py-libp2p.readthedocs.io <https://py-libp2p.readthedocs.io>`_ (`#318 <https://github.com/libp2p/py-libp2p/issues/318>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Added Makefile target to test a packaged version of libp2p before release. (`#353 <https://github.com/libp2p/py-libp2p/issues/353>`__)
- Move helper tools from ``tests/`` to ``libp2p/tools/``, and some mildly-related cleanups. (`#356 <https://github.com/libp2p/py-libp2p/issues/356>`__)
Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~
- `#357 <https://github.com/libp2p/py-libp2p/issues/357>`__
v0.1.2
--------------
Welcome to the great beyond, where changes were not tracked by release...

View File

@ -1,92 +1,103 @@
import asyncio
import argparse
import sys
import click
import multiaddr
import trio
from libp2p import new_node
from libp2p import new_host
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.typing import TProtocol
PROTOCOL_ID = TProtocol("/chat/1.0.0")
MAX_READ_LEN = 2 ** 32 - 1
PROTOCOL_ID = '/chat/1.0.0'
async def read_data(stream):
async def read_data(stream: INetStream) -> None:
while True:
read_string = await stream.read()
if read_string is not None:
read_string = read_string.decode()
read_bytes = await stream.read(MAX_READ_LEN)
if read_bytes is not None:
read_string = read_bytes.decode()
if read_string != "\n":
# Green console colour: \x1b[32m
# Reset console colour: \x1b[0m
print("\x1b[32m %s\x1b[0m " % read_string, end="")
async def write_data(stream):
loop = asyncio.get_event_loop()
async def write_data(stream: INetStream) -> None:
async_f = trio.wrap_file(sys.stdin)
while True:
line = await loop.run_in_executor(None, sys.stdin.readline)
line = await async_f.readline()
await stream.write(line.encode())
async def run(port, destination):
host = await new_node(transport_opt=["/ip4/127.0.0.1/tcp/%s" % port])
if not destination: # its the server
async def stream_handler(stream):
asyncio.ensure_future(read_data(stream))
asyncio.ensure_future(write_data(stream))
host.set_stream_handler(PROTOCOL_ID, stream_handler)
async def run(port: int, destination: str) -> None:
localhost_ip = "127.0.0.1"
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
host = new_host()
async with host.run(listen_addrs=[listen_addr]), trio.open_nursery() as nursery:
if not destination: # its the server
port = None
for listener in host.network.listeners.values():
for addr in listener.get_addrs():
port = int(addr.value_for_protocol('tcp'))
async def stream_handler(stream: INetStream) -> None:
nursery.start_soon(read_data, stream)
nursery.start_soon(write_data, stream)
if not port:
raise RuntimeError("was not able to find the actual local port")
host.set_stream_handler(PROTOCOL_ID, stream_handler)
print("Run './examples/chat/chat.py -p %s -d /ip4/127.0.0.1/tcp/%s/p2p/%s' on another console.\n" %
(int(port) + 1, port, host.get_id().pretty()))
print("You can replace 127.0.0.1 with public IP as well.")
print("\nWaiting for incoming connection\n\n")
print(
f"Run 'python ./examples/chat/chat.py "
f"-p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id().pretty()}' "
"on another console."
)
print("Waiting for incoming connection...")
else: # its the client
m = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(m)
# Associate the peer with local ip address
await host.connect(info)
else: # its the client
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
# Associate the peer with local ip address
await host.connect(info)
# Start a stream with the destination.
# Multiaddress of the destination peer is fetched from the peerstore using 'peerId'.
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
# Start a stream with the destination.
# Multiaddress of the destination peer is fetched from the peerstore using 'peerId'.
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
nursery.start_soon(read_data, stream)
nursery.start_soon(write_data, stream)
print(f"Connected to peer {info.addrs[0]}")
asyncio.ensure_future(read_data(stream))
asyncio.ensure_future(write_data(stream))
print("Connected to peer %s" % info.addrs[0])
await trio.sleep_forever()
@click.command()
@click.option('--port', '-p', help='source port number', default=8000)
@click.option('--destination', '-d', help="Destination multiaddr string")
@click.option('--help', is_flag=True, default=False, help='display help')
# @click.option('--debug', is_flag=True, default=False, help='Debug generates the same node ID on every execution')
def main(port, destination, help):
def main() -> None:
description = """
This program demonstrates a simple p2p chat application using libp2p.
To use it, first run 'python ./chat -p <PORT>', where <PORT> is the port number.
Then, run another host with 'python ./chat -p <ANOTHER_PORT> -d <DESTINATION>',
where <DESTINATION> is the multiaddress of the previous listener host.
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p", "--port", default=8000, type=int, help="source port number"
)
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
args = parser.parse_args()
if help:
print("This program demonstrates a simple p2p chat application using libp2p\n\n")
print("Usage: Run './chat -p <SOURCE_PORT>' where <SOURCE_PORT> can be any port number.")
print("Now run './chat -p <PORT> -d <MULTIADDR>' where <MULTIADDR> is multiaddress of previous listener host.")
return
if not args.port:
raise RuntimeError("was not able to determine a local port")
loop = asyncio.get_event_loop()
try:
asyncio.ensure_future(run(port, destination))
loop.run_forever()
trio.run(run, *(args.port, args.destination))
except KeyboardInterrupt:
pass
finally:
loop.close()
if __name__ == '__main__':
if __name__ == "__main__":
main()

116
examples/echo/echo.py Normal file
View File

@ -0,0 +1,116 @@
import argparse
import multiaddr
import trio
from libp2p import new_host
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.typing import TProtocol
PROTOCOL_ID = TProtocol("/echo/1.0.0")
async def _echo_stream_handler(stream: INetStream) -> None:
# Wait until EOF
msg = await stream.read()
await stream.write(msg)
await stream.close()
async def run(port: int, destination: str, seed: int = None) -> None:
localhost_ip = "127.0.0.1"
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
if seed:
import random
random.seed(seed)
secret_number = random.getrandbits(32 * 8)
secret = secret_number.to_bytes(length=32, byteorder="big")
else:
import secrets
secret = secrets.token_bytes(32)
host = new_host(key_pair=create_new_key_pair(secret))
async with host.run(listen_addrs=[listen_addr]):
print(f"I am {host.get_id().to_string()}")
if not destination: # its the server
host.set_stream_handler(PROTOCOL_ID, _echo_stream_handler)
print(
f"Run 'python ./examples/echo/echo.py "
f"-p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id().pretty()}' "
"on another console."
)
print("Waiting for incoming connections...")
await trio.sleep_forever()
else: # its the client
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
# Associate the peer with local ip address
await host.connect(info)
# Start a stream with the destination.
# Multiaddress of the destination peer is fetched from the peerstore using 'peerId'.
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
msg = b"hi, there!\n"
await stream.write(msg)
# Notify the other side about EOF
await stream.close()
response = await stream.read()
print(f"Sent: {msg}")
print(f"Got: {response}")
def main() -> None:
description = """
This program demonstrates a simple echo protocol where a peer listens for
connections and copies back any input received on a stream.
To use it, first run 'python ./echo -p <PORT>', where <PORT> is the port number.
Then, run another host with 'python ./chat -p <ANOTHER_PORT> -d <DESTINATION>',
where <DESTINATION> is the multiaddress of the previous listener host.
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p", "--port", default=8000, type=int, help="source port number"
)
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
parser.add_argument(
"-s",
"--seed",
type=int,
help="provide a seed to the random number generator (e.g. to fix peer IDs across runs)",
)
args = parser.parse_args()
if not args.port:
raise RuntimeError("was not able to determine a local port")
try:
trio.run(run, args.port, args.destination, args.seed)
except KeyboardInterrupt:
pass
if __name__ == "__main__":
main()

View File

@ -1,38 +1,98 @@
from Crypto.PublicKey import RSA
import multiaddr
from .peer.peerstore import PeerStore
from .peer.id import id_from_public_key
from .network.swarm import Swarm
from .host.basic_host import BasicHost
from .transport.upgrader import TransportUpgrader
from .transport.tcp.tcp import TCP
from libp2p.crypto.keys import KeyPair
from libp2p.crypto.rsa import create_new_key_pair
from libp2p.host.basic_host import BasicHost
from libp2p.host.host_interface import IHost
from libp2p.host.routed_host import RoutedHost
from libp2p.network.network_interface import INetworkService
from libp2p.network.swarm import Swarm
from libp2p.peer.id import ID
from libp2p.peer.peerstore import PeerStore
from libp2p.peer.peerstore_interface import IPeerStore
from libp2p.routing.interfaces import IPeerRouting
from libp2p.security.insecure.transport import PLAINTEXT_PROTOCOL_ID, InsecureTransport
import libp2p.security.secio.transport as secio
from libp2p.stream_muxer.mplex.mplex import MPLEX_PROTOCOL_ID, Mplex
from libp2p.transport.tcp.tcp import TCP
from libp2p.transport.typing import TMuxerOptions, TSecurityOptions
from libp2p.transport.upgrader import TransportUpgrader
from libp2p.typing import TProtocol
async def new_node(
id_opt=None, transport_opt=None,
muxer_opt=None, sec_opt=None, peerstore=None):
def generate_new_rsa_identity() -> KeyPair:
return create_new_key_pair()
if id_opt is None:
new_key = RSA.generate(2048, e=65537)
id_opt = id_from_public_key(new_key.publickey())
# private_key = new_key.exportKey("PEM")
transport_opt = transport_opt or ["/ip4/127.0.0.1/tcp/8001"]
transport_opt = [multiaddr.Multiaddr(t) for t in transport_opt]
muxer_opt = muxer_opt or ["mplex/6.7.0"]
sec_opt = sec_opt or ["secio"]
peerstore = peerstore or PeerStore()
def generate_peer_id_from(key_pair: KeyPair) -> ID:
public_key = key_pair.public_key
return ID.from_pubkey(public_key)
upgrader = TransportUpgrader(sec_opt, transport_opt)
swarm = Swarm(id_opt, peerstore, upgrader)
tcp = TCP()
swarm.add_transport(tcp)
await swarm.listen(transport_opt[0])
# TODO enable support for other host type
# TODO routing unimplemented
host = BasicHost(swarm)
def new_swarm(
key_pair: KeyPair = None,
muxer_opt: TMuxerOptions = None,
sec_opt: TSecurityOptions = None,
peerstore_opt: IPeerStore = None,
) -> INetworkService:
"""
Create a swarm instance based on the parameters.
:param key_pair: optional choice of the ``KeyPair``
:param muxer_opt: optional choice of stream muxer
:param sec_opt: optional choice of security upgrade
:param peerstore_opt: optional peerstore
:return: return a default swarm instance
"""
if key_pair is None:
key_pair = generate_new_rsa_identity()
id_opt = generate_peer_id_from(key_pair)
# TODO: Parse `listen_addrs` to determine transport
transport = TCP()
muxer_transports_by_protocol = muxer_opt or {MPLEX_PROTOCOL_ID: Mplex}
security_transports_by_protocol = sec_opt or {
TProtocol(PLAINTEXT_PROTOCOL_ID): InsecureTransport(key_pair),
TProtocol(secio.ID): secio.Transport(key_pair),
}
upgrader = TransportUpgrader(
security_transports_by_protocol, muxer_transports_by_protocol
)
peerstore = peerstore_opt or PeerStore()
# Store our key pair in peerstore
peerstore.add_key_pair(id_opt, key_pair)
return Swarm(id_opt, peerstore, upgrader, transport)
def new_host(
key_pair: KeyPair = None,
muxer_opt: TMuxerOptions = None,
sec_opt: TSecurityOptions = None,
peerstore_opt: IPeerStore = None,
disc_opt: IPeerRouting = None,
) -> IHost:
"""
Create a new libp2p host based on the given parameters.
:param key_pair: optional choice of the ``KeyPair``
:param muxer_opt: optional choice of stream muxer
:param sec_opt: optional choice of security upgrade
:param peerstore_opt: optional peerstore
:param disc_opt: optional discovery
:return: return a host instance
"""
swarm = new_swarm(
key_pair=key_pair,
muxer_opt=muxer_opt,
sec_opt=sec_opt,
peerstore_opt=peerstore_opt,
)
host: IHost
if disc_opt:
host = RoutedHost(swarm, disc_opt)
else:
host = BasicHost(swarm)
return host

View File

@ -0,0 +1,125 @@
from dataclasses import dataclass
import hmac
from typing import Tuple
from Crypto.Cipher import AES
import Crypto.Util.Counter as Counter
class InvalidMACException(Exception):
pass
@dataclass(frozen=True)
class EncryptionParameters:
cipher_type: str
hash_type: str
iv: bytes
mac_key: bytes
cipher_key: bytes
class MacAndCipher:
def __init__(self, parameters: EncryptionParameters) -> None:
self.authenticator = hmac.new(
parameters.mac_key, digestmod=parameters.hash_type
)
iv_bit_size = 8 * len(parameters.iv)
cipher = AES.new(
parameters.cipher_key,
AES.MODE_CTR,
counter=Counter.new(
iv_bit_size,
initial_value=int.from_bytes(parameters.iv, byteorder="big"),
),
)
self.cipher = cipher
def encrypt(self, data: bytes) -> bytes:
return self.cipher.encrypt(data)
def authenticate(self, data: bytes) -> bytes:
authenticator = self.authenticator.copy()
authenticator.update(data)
return authenticator.digest()
def decrypt_if_valid(self, data_with_tag: bytes) -> bytes:
tag_position = len(data_with_tag) - self.authenticator.digest_size
data = data_with_tag[:tag_position]
tag = data_with_tag[tag_position:]
authenticator = self.authenticator.copy()
authenticator.update(data)
expected_tag = authenticator.digest()
if not hmac.compare_digest(tag, expected_tag):
raise InvalidMACException(expected_tag, tag)
return self.cipher.decrypt(data)
def initialize_pair(
cipher_type: str, hash_type: str, secret: bytes
) -> Tuple[EncryptionParameters, EncryptionParameters]:
"""Return a pair of ``Keys`` for use in securing a communications channel
with authenticated encryption derived from the ``secret`` and using the
requested ``cipher_type`` and ``hash_type``."""
if cipher_type != "AES-128":
raise NotImplementedError()
if hash_type != "SHA256":
raise NotImplementedError()
iv_size = 16
cipher_key_size = 16
hmac_key_size = 20
seed = "key expansion".encode()
params_size = iv_size + cipher_key_size + hmac_key_size
result = bytearray(2 * params_size)
authenticator = hmac.new(secret, digestmod=hash_type)
authenticator.update(seed)
tag = authenticator.digest()
i = 0
len_result = 2 * params_size
while i < len_result:
authenticator = hmac.new(secret, digestmod=hash_type)
authenticator.update(tag)
authenticator.update(seed)
another_tag = authenticator.digest()
remaining_bytes = len(another_tag)
if i + remaining_bytes > len_result:
remaining_bytes = len_result - i
result[i : i + remaining_bytes] = another_tag[0:remaining_bytes]
i += remaining_bytes
authenticator = hmac.new(secret, digestmod=hash_type)
authenticator.update(tag)
tag = authenticator.digest()
first_half = result[:params_size]
second_half = result[params_size:]
return (
EncryptionParameters(
cipher_type,
hash_type,
first_half[0:iv_size],
first_half[iv_size + cipher_key_size :],
first_half[iv_size : iv_size + cipher_key_size],
),
EncryptionParameters(
cipher_type,
hash_type,
second_half[0:iv_size],
second_half[iv_size + cipher_key_size :],
second_half[iv_size : iv_size + cipher_key_size],
),
)

68
libp2p/crypto/ecc.py Normal file
View File

@ -0,0 +1,68 @@
from fastecdsa import curve as curve_types
from fastecdsa import keys, point
from fastecdsa.encoding.sec1 import SEC1Encoder
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
def infer_local_type(curve: str) -> curve_types.Curve:
"""converts a ``str`` representation of some elliptic curve to a
representation understood by the backend of this module."""
if curve == "P-256":
return curve_types.P256
else:
raise NotImplementedError()
class ECCPublicKey(PublicKey):
def __init__(self, impl: point.Point, curve: curve_types.Curve) -> None:
self.impl = impl
self.curve = curve
def to_bytes(self) -> bytes:
return SEC1Encoder.encode_public_key(self.impl, compressed=False)
@classmethod
def from_bytes(cls, data: bytes, curve: str) -> "ECCPublicKey":
curve_type = infer_local_type(curve)
public_key_impl = SEC1Encoder.decode_public_key(data, curve_type)
return cls(public_key_impl, curve_type)
def get_type(self) -> KeyType:
return KeyType.ECC_P256
def verify(self, data: bytes, signature: bytes) -> bool:
raise NotImplementedError()
class ECCPrivateKey(PrivateKey):
def __init__(self, impl: int, curve: curve_types.Curve) -> None:
self.impl = impl
self.curve = curve
@classmethod
def new(cls, curve: str) -> "ECCPrivateKey":
curve_type = infer_local_type(curve)
private_key_impl = keys.gen_private_key(curve_type)
return cls(private_key_impl, curve_type)
def to_bytes(self) -> bytes:
return keys.export_key(self.impl, self.curve)
def get_type(self) -> KeyType:
return KeyType.ECC_P256
def sign(self, data: bytes) -> bytes:
raise NotImplementedError()
def get_public_key(self) -> PublicKey:
public_key_impl = keys.get_public_key(self.impl, self.curve)
return ECCPublicKey(public_key_impl, self.curve)
def create_new_key_pair(curve: str) -> KeyPair:
"""Return a new ECC keypair with the requested ``curve`` type, e.g.
"P-256"."""
private_key = ECCPrivateKey.new(curve)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

69
libp2p/crypto/ed25519.py Normal file
View File

@ -0,0 +1,69 @@
from Crypto.Hash import SHA256
from nacl.exceptions import BadSignatureError
from nacl.public import PrivateKey as PrivateKeyImpl
from nacl.public import PublicKey as PublicKeyImpl
from nacl.signing import SigningKey, VerifyKey
import nacl.utils as utils
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
class Ed25519PublicKey(PublicKey):
def __init__(self, impl: PublicKeyImpl) -> None:
self.impl = impl
def to_bytes(self) -> bytes:
return bytes(self.impl)
@classmethod
def from_bytes(cls, key_bytes: bytes) -> "Ed25519PublicKey":
return cls(PublicKeyImpl(key_bytes))
def get_type(self) -> KeyType:
return KeyType.Ed25519
def verify(self, data: bytes, signature: bytes) -> bool:
verify_key = VerifyKey(self.to_bytes())
try:
verify_key.verify(data, signature)
except BadSignatureError:
return False
return True
class Ed25519PrivateKey(PrivateKey):
def __init__(self, impl: PrivateKeyImpl) -> None:
self.impl = impl
@classmethod
def new(cls, seed: bytes = None) -> "Ed25519PrivateKey":
if not seed:
seed = utils.random()
private_key_impl = PrivateKeyImpl.from_seed(seed)
return cls(private_key_impl)
def to_bytes(self) -> bytes:
return bytes(self.impl)
@classmethod
def from_bytes(cls, data: bytes) -> "Ed25519PrivateKey":
impl = PrivateKeyImpl(data)
return cls(impl)
def get_type(self) -> KeyType:
return KeyType.Ed25519
def sign(self, data: bytes) -> bytes:
h = SHA256.new(data)
signing_key = SigningKey(self.to_bytes())
return signing_key.sign(h)
def get_public_key(self) -> PublicKey:
return Ed25519PublicKey(self.impl.public_key)
def create_new_key_pair(seed: bytes = None) -> KeyPair:
private_key = Ed25519PrivateKey.new(seed)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

View File

@ -0,0 +1,12 @@
from libp2p.exceptions import BaseLibp2pError
class CryptographyError(BaseLibp2pError):
pass
class MissingDeserializerError(CryptographyError):
"""Raise if the requested deserialization routine is missing for some type
of cryptographic key."""
pass

View File

@ -0,0 +1,29 @@
from typing import Callable, Tuple, cast
from fastecdsa.encoding import util
from libp2p.crypto.ecc import ECCPrivateKey, ECCPublicKey, create_new_key_pair
from libp2p.crypto.keys import PublicKey
SharedKeyGenerator = Callable[[bytes], bytes]
int_bytelen = util.int_bytelen
def create_ephemeral_key_pair(curve_type: str) -> Tuple[PublicKey, SharedKeyGenerator]:
"""Facilitates ECDH key exchange."""
if curve_type != "P-256":
raise NotImplementedError()
key_pair = create_new_key_pair(curve_type)
def _key_exchange(serialized_remote_public_key: bytes) -> bytes:
private_key = cast(ECCPrivateKey, key_pair.private_key)
remote_point = ECCPublicKey.from_bytes(serialized_remote_public_key, curve_type)
secret_point = remote_point.impl * private_key.impl
secret_x_coordinate = secret_point.x
byte_size = int_bytelen(secret_x_coordinate)
return secret_x_coordinate.to_bytes(byte_size, byteorder="big")
return key_pair.public_key, _key_exchange

91
libp2p/crypto/keys.py Normal file
View File

@ -0,0 +1,91 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
from enum import Enum, unique
from .pb import crypto_pb2 as protobuf
@unique
class KeyType(Enum):
RSA = 0
Ed25519 = 1
Secp256k1 = 2
ECDSA = 3
ECC_P256 = 4
class Key(ABC):
"""A ``Key`` represents a cryptographic key."""
@abstractmethod
def to_bytes(self) -> bytes:
"""Returns the byte representation of this key."""
...
@abstractmethod
def get_type(self) -> KeyType:
"""Returns the ``KeyType`` for ``self``."""
...
def __eq__(self, other: object) -> bool:
if not isinstance(other, Key):
return NotImplemented
return self.to_bytes() == other.to_bytes()
class PublicKey(Key):
"""A ``PublicKey`` represents a cryptographic public key."""
@abstractmethod
def verify(self, data: bytes, signature: bytes) -> bool:
"""Verify that ``signature`` is the cryptographic signature of the hash
of ``data``."""
...
def _serialize_to_protobuf(self) -> protobuf.PublicKey:
"""Return the protobuf representation of this ``Key``."""
key_type = self.get_type().value
data = self.to_bytes()
protobuf_key = protobuf.PublicKey(key_type=key_type, data=data)
return protobuf_key
def serialize(self) -> bytes:
"""Return the canonical serialization of this ``Key``."""
return self._serialize_to_protobuf().SerializeToString()
@classmethod
def deserialize_from_protobuf(cls, protobuf_data: bytes) -> protobuf.PublicKey:
return protobuf.PublicKey.FromString(protobuf_data)
class PrivateKey(Key):
"""A ``PrivateKey`` represents a cryptographic private key."""
@abstractmethod
def sign(self, data: bytes) -> bytes:
...
@abstractmethod
def get_public_key(self) -> PublicKey:
...
def _serialize_to_protobuf(self) -> protobuf.PrivateKey:
"""Return the protobuf representation of this ``Key``."""
key_type = self.get_type().value
data = self.to_bytes()
protobuf_key = protobuf.PrivateKey(key_type=key_type, data=data)
return protobuf_key
def serialize(self) -> bytes:
"""Return the canonical serialization of this ``Key``."""
return self._serialize_to_protobuf().SerializeToString()
@classmethod
def deserialize_from_protobuf(cls, protobuf_data: bytes) -> protobuf.PrivateKey:
return protobuf.PrivateKey.FromString(protobuf_data)
@dataclass(frozen=True)
class KeyPair:
private_key: PrivateKey
public_key: PublicKey

View File

@ -0,0 +1,20 @@
syntax = "proto2";
package crypto.pb;
enum KeyType {
RSA = 0;
Ed25519 = 1;
Secp256k1 = 2;
ECDSA = 3;
}
message PublicKey {
required KeyType key_type = 1;
required bytes data = 2;
}
message PrivateKey {
required KeyType key_type = 1;
required bytes data = 2;
}

View File

@ -0,0 +1,162 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: libp2p/crypto/pb/crypto.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf.internal import enum_type_wrapper
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='libp2p/crypto/pb/crypto.proto',
package='crypto.pb',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n\x1dlibp2p/crypto/pb/crypto.proto\x12\tcrypto.pb\"?\n\tPublicKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c\"@\n\nPrivateKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c*9\n\x07KeyType\x12\x07\n\x03RSA\x10\x00\x12\x0b\n\x07\x45\x64\x32\x35\x35\x31\x39\x10\x01\x12\r\n\tSecp256k1\x10\x02\x12\t\n\x05\x45\x43\x44SA\x10\x03')
)
_KEYTYPE = _descriptor.EnumDescriptor(
name='KeyType',
full_name='crypto.pb.KeyType',
filename=None,
file=DESCRIPTOR,
values=[
_descriptor.EnumValueDescriptor(
name='RSA', index=0, number=0,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='Ed25519', index=1, number=1,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='Secp256k1', index=2, number=2,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='ECDSA', index=3, number=3,
serialized_options=None,
type=None),
],
containing_type=None,
serialized_options=None,
serialized_start=175,
serialized_end=232,
)
_sym_db.RegisterEnumDescriptor(_KEYTYPE)
KeyType = enum_type_wrapper.EnumTypeWrapper(_KEYTYPE)
RSA = 0
Ed25519 = 1
Secp256k1 = 2
ECDSA = 3
_PUBLICKEY = _descriptor.Descriptor(
name='PublicKey',
full_name='crypto.pb.PublicKey',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='key_type', full_name='crypto.pb.PublicKey.key_type', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='data', full_name='crypto.pb.PublicKey.data', index=1,
number=2, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto2',
extension_ranges=[],
oneofs=[
],
serialized_start=44,
serialized_end=107,
)
_PRIVATEKEY = _descriptor.Descriptor(
name='PrivateKey',
full_name='crypto.pb.PrivateKey',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='key_type', full_name='crypto.pb.PrivateKey.key_type', index=0,
number=1, type=14, cpp_type=8, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='data', full_name='crypto.pb.PrivateKey.data', index=1,
number=2, type=12, cpp_type=9, label=2,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto2',
extension_ranges=[],
oneofs=[
],
serialized_start=109,
serialized_end=173,
)
_PUBLICKEY.fields_by_name['key_type'].enum_type = _KEYTYPE
_PRIVATEKEY.fields_by_name['key_type'].enum_type = _KEYTYPE
DESCRIPTOR.message_types_by_name['PublicKey'] = _PUBLICKEY
DESCRIPTOR.message_types_by_name['PrivateKey'] = _PRIVATEKEY
DESCRIPTOR.enum_types_by_name['KeyType'] = _KEYTYPE
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
PublicKey = _reflection.GeneratedProtocolMessageType('PublicKey', (_message.Message,), {
'DESCRIPTOR' : _PUBLICKEY,
'__module__' : 'libp2p.crypto.pb.crypto_pb2'
# @@protoc_insertion_point(class_scope:crypto.pb.PublicKey)
})
_sym_db.RegisterMessage(PublicKey)
PrivateKey = _reflection.GeneratedProtocolMessageType('PrivateKey', (_message.Message,), {
'DESCRIPTOR' : _PRIVATEKEY,
'__module__' : 'libp2p.crypto.pb.crypto_pb2'
# @@protoc_insertion_point(class_scope:crypto.pb.PrivateKey)
})
_sym_db.RegisterMessage(PrivateKey)
# @@protoc_insertion_point(module_scope)

View File

@ -0,0 +1,84 @@
# @generated by generate_proto_mypy_stubs.py. Do not edit!
import sys
from google.protobuf.descriptor import (
Descriptor as google___protobuf___descriptor___Descriptor,
EnumDescriptor as google___protobuf___descriptor___EnumDescriptor,
)
from google.protobuf.message import (
Message as google___protobuf___message___Message,
)
from typing import (
List as typing___List,
Tuple as typing___Tuple,
cast as typing___cast,
)
from typing_extensions import (
Literal as typing_extensions___Literal,
)
class KeyType(int):
DESCRIPTOR: google___protobuf___descriptor___EnumDescriptor = ...
@classmethod
def Name(cls, number: int) -> str: ...
@classmethod
def Value(cls, name: str) -> KeyType: ...
@classmethod
def keys(cls) -> typing___List[str]: ...
@classmethod
def values(cls) -> typing___List[KeyType]: ...
@classmethod
def items(cls) -> typing___List[typing___Tuple[str, KeyType]]: ...
RSA = typing___cast(KeyType, 0)
Ed25519 = typing___cast(KeyType, 1)
Secp256k1 = typing___cast(KeyType, 2)
ECDSA = typing___cast(KeyType, 3)
RSA = typing___cast(KeyType, 0)
Ed25519 = typing___cast(KeyType, 1)
Secp256k1 = typing___cast(KeyType, 2)
ECDSA = typing___cast(KeyType, 3)
class PublicKey(google___protobuf___message___Message):
DESCRIPTOR: google___protobuf___descriptor___Descriptor = ...
key_type = ... # type: KeyType
data = ... # type: bytes
def __init__(self,
*,
key_type : KeyType,
data : bytes,
) -> None: ...
@classmethod
def FromString(cls, s: bytes) -> PublicKey: ...
def MergeFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
def CopyFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
if sys.version_info >= (3,):
def HasField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> None: ...
else:
def HasField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> None: ...
class PrivateKey(google___protobuf___message___Message):
DESCRIPTOR: google___protobuf___descriptor___Descriptor = ...
key_type = ... # type: KeyType
data = ... # type: bytes
def __init__(self,
*,
key_type : KeyType,
data : bytes,
) -> None: ...
@classmethod
def FromString(cls, s: bytes) -> PrivateKey: ...
def MergeFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
def CopyFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
if sys.version_info >= (3,):
def HasField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",u"key_type"]) -> None: ...
else:
def HasField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"data",b"data",u"key_type",b"key_type"]) -> None: ...

65
libp2p/crypto/rsa.py Normal file
View File

@ -0,0 +1,65 @@
from Crypto.Hash import SHA256
import Crypto.PublicKey.RSA as RSA
from Crypto.PublicKey.RSA import RsaKey
from Crypto.Signature import pkcs1_15
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
class RSAPublicKey(PublicKey):
def __init__(self, impl: RsaKey) -> None:
self.impl = impl
def to_bytes(self) -> bytes:
return self.impl.export_key("DER")
@classmethod
def from_bytes(cls, key_bytes: bytes) -> "RSAPublicKey":
rsakey = RSA.import_key(key_bytes)
return cls(rsakey)
def get_type(self) -> KeyType:
return KeyType.RSA
def verify(self, data: bytes, signature: bytes) -> bool:
h = SHA256.new(data)
try:
pkcs1_15.new(self.impl).verify(h, signature)
except (ValueError, TypeError):
return False
return True
class RSAPrivateKey(PrivateKey):
def __init__(self, impl: RsaKey) -> None:
self.impl = impl
@classmethod
def new(cls, bits: int = 2048, e: int = 65537) -> "RSAPrivateKey":
private_key_impl = RSA.generate(bits, e=e)
return cls(private_key_impl)
def to_bytes(self) -> bytes:
return self.impl.export_key("DER")
def get_type(self) -> KeyType:
return KeyType.RSA
def sign(self, data: bytes) -> bytes:
h = SHA256.new(data)
return pkcs1_15.new(self.impl).sign(h)
def get_public_key(self) -> PublicKey:
return RSAPublicKey(self.impl.publickey())
def create_new_key_pair(bits: int = 2048, e: int = 65537) -> KeyPair:
"""
Returns a new RSA keypair with the requested key size (``bits``) and the
given public exponent ``e``.
Sane defaults are provided for both values.
"""
private_key = RSAPrivateKey.new(bits, e)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

View File

@ -0,0 +1,73 @@
import coincurve
from libp2p.crypto.keys import KeyPair, KeyType, PrivateKey, PublicKey
class Secp256k1PublicKey(PublicKey):
def __init__(self, impl: coincurve.PublicKey) -> None:
self.impl = impl
def to_bytes(self) -> bytes:
return self.impl.format()
@classmethod
def from_bytes(cls, data: bytes) -> "Secp256k1PublicKey":
impl = coincurve.PublicKey(data)
return cls(impl)
@classmethod
def deserialize(cls, data: bytes) -> "Secp256k1PublicKey":
protobuf_key = cls.deserialize_from_protobuf(data)
return cls.from_bytes(protobuf_key.data)
def get_type(self) -> KeyType:
return KeyType.Secp256k1
def verify(self, data: bytes, signature: bytes) -> bool:
return self.impl.verify(signature, data)
class Secp256k1PrivateKey(PrivateKey):
def __init__(self, impl: coincurve.PrivateKey) -> None:
self.impl = impl
@classmethod
def new(cls, secret: bytes = None) -> "Secp256k1PrivateKey":
private_key_impl = coincurve.PrivateKey(secret)
return cls(private_key_impl)
def to_bytes(self) -> bytes:
return self.impl.secret
@classmethod
def from_bytes(cls, data: bytes) -> "Secp256k1PrivateKey":
impl = coincurve.PrivateKey(data)
return cls(impl)
@classmethod
def deserialize(cls, data: bytes) -> "Secp256k1PrivateKey":
protobuf_key = cls.deserialize_from_protobuf(data)
return cls.from_bytes(protobuf_key.data)
def get_type(self) -> KeyType:
return KeyType.Secp256k1
def sign(self, data: bytes) -> bytes:
return self.impl.sign(data)
def get_public_key(self) -> PublicKey:
public_key_impl = coincurve.PublicKey.from_secret(self.impl.secret)
return Secp256k1PublicKey(public_key_impl)
def create_new_key_pair(secret: bytes = None) -> KeyPair:
"""
Returns a new Secp256k1 keypair derived from the provided ``secret``, a
sequence of bytes corresponding to some integer between 0 and the group
order.
A valid secret is created if ``None`` is passed.
"""
private_key = Secp256k1PrivateKey.new(secret)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

View File

@ -0,0 +1,38 @@
from libp2p.crypto.ed25519 import Ed25519PrivateKey, Ed25519PublicKey
from libp2p.crypto.exceptions import MissingDeserializerError
from libp2p.crypto.keys import KeyType, PrivateKey, PublicKey
from libp2p.crypto.rsa import RSAPublicKey
from libp2p.crypto.secp256k1 import Secp256k1PrivateKey, Secp256k1PublicKey
key_type_to_public_key_deserializer = {
KeyType.Secp256k1.value: Secp256k1PublicKey.from_bytes,
KeyType.RSA.value: RSAPublicKey.from_bytes,
KeyType.Ed25519.value: Ed25519PublicKey.from_bytes,
}
key_type_to_private_key_deserializer = {
KeyType.Secp256k1.value: Secp256k1PrivateKey.from_bytes,
KeyType.Ed25519.value: Ed25519PrivateKey.from_bytes,
}
def deserialize_public_key(data: bytes) -> PublicKey:
f = PublicKey.deserialize_from_protobuf(data)
try:
deserializer = key_type_to_public_key_deserializer[f.key_type]
except KeyError as e:
raise MissingDeserializerError(
{"key_type": f.key_type, "key": "public_key"}
) from e
return deserializer(f.data)
def deserialize_private_key(data: bytes) -> PrivateKey:
f = PrivateKey.deserialize_from_protobuf(data)
try:
deserializer = key_type_to_private_key_deserializer[f.key_type]
except KeyError as e:
raise MissingDeserializerError(
{"key_type": f.key_type, "key": "private_key"}
) from e
return deserializer(f.data)

16
libp2p/exceptions.py Normal file
View File

@ -0,0 +1,16 @@
class BaseLibp2pError(Exception):
pass
class ValidationError(BaseLibp2pError):
"""Raised when something does not pass a validation check."""
class ParseError(BaseLibp2pError):
pass
class MultiError(BaseLibp2pError):
"""Raised with multiple exceptions."""
# todo: find some way for this to fancy-print all encapsulated errors

View File

@ -1,90 +1,195 @@
import logging
from typing import TYPE_CHECKING, AsyncIterator, List, Sequence
from async_generator import asynccontextmanager
from async_service import background_trio_service
import multiaddr
from libp2p.crypto.keys import PrivateKey, PublicKey
from libp2p.host.defaults import get_default_protocols
from libp2p.host.exceptions import StreamFailure
from libp2p.network.network_interface import INetworkService
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import PeerInfo
from libp2p.peer.peerstore_interface import IPeerStore
from libp2p.protocol_muxer.exceptions import MultiselectClientError, MultiselectError
from libp2p.protocol_muxer.multiselect import Multiselect
from libp2p.protocol_muxer.multiselect_client import MultiselectClient
from libp2p.protocol_muxer.multiselect_communicator import MultiselectCommunicator
from libp2p.typing import StreamHandlerFn, TProtocol
from .host_interface import IHost
if TYPE_CHECKING:
from collections import OrderedDict
# Upon host creation, host takes in options,
# including the list of addresses on which to listen.
# Host then parses these options and delegates to its Network instance,
# telling it to listen on the given listen addresses.
logger = logging.getLogger("libp2p.network.basic_host")
class BasicHost(IHost):
"""
BasicHost is a wrapper of a `INetwork` implementation.
# default options constructor
def __init__(self, _network):
self.network = _network
self.peerstore = self.network.peerstore
It performs protocol negotiation on a stream with multistream-select
right after a stream is initialized.
"""
def get_id(self):
_network: INetworkService
peerstore: IPeerStore
multiselect: Multiselect
multiselect_client: MultiselectClient
def __init__(
self,
network: INetworkService,
default_protocols: "OrderedDict[TProtocol, StreamHandlerFn]" = None,
) -> None:
self._network = network
self._network.set_stream_handler(self._swarm_stream_handler)
self.peerstore = self._network.peerstore
# Protocol muxing
default_protocols = default_protocols or get_default_protocols(self)
self.multiselect = Multiselect(default_protocols)
self.multiselect_client = MultiselectClient()
def get_id(self) -> ID:
"""
:return: peer_id of host
"""
return self.network.get_peer_id()
return self._network.get_peer_id()
def get_network(self):
def get_public_key(self) -> PublicKey:
return self.peerstore.pubkey(self.get_id())
def get_private_key(self) -> PrivateKey:
return self.peerstore.privkey(self.get_id())
def get_network(self) -> INetworkService:
"""
:return: network instance of host
"""
return self.network
return self._network
def get_peerstore(self):
def get_peerstore(self) -> IPeerStore:
"""
:return: peerstore of the host (same one as in its network instance)
"""
return self.peerstore
def get_mux(self):
def get_mux(self) -> Multiselect:
"""
:return: mux instance of host
"""
return self.multiselect
def get_addrs(self):
def get_addrs(self) -> List[multiaddr.Multiaddr]:
"""
:return: all the multiaddr addresses this host is listening too
:return: all the multiaddr addresses this host is listening to
"""
p2p_part = multiaddr.Multiaddr('/p2p/{}'.format(self.get_id().pretty()))
# TODO: We don't need "/p2p/{peer_id}" postfix actually.
p2p_part = multiaddr.Multiaddr(f"/p2p/{self.get_id()!s}")
addrs = []
for transport in self.network.listeners.values():
addrs: List[multiaddr.Multiaddr] = []
for transport in self._network.listeners.values():
for addr in transport.get_addrs():
addrs.append(addr.encapsulate(p2p_part))
return addrs
def set_stream_handler(self, protocol_id, stream_handler):
@asynccontextmanager
async def run(
self, listen_addrs: Sequence[multiaddr.Multiaddr]
) -> AsyncIterator[None]:
"""
set stream handler for host
run the host instance and listen to ``listen_addrs``.
:param listen_addrs: a sequence of multiaddrs that we want to listen to
"""
network = self.get_network()
async with background_trio_service(network):
await network.listen(*listen_addrs)
yield
def set_stream_handler(
self, protocol_id: TProtocol, stream_handler: StreamHandlerFn
) -> None:
"""
set stream handler for given `protocol_id`
:param protocol_id: protocol id used on stream
:param stream_handler: a stream handler function
:return: true if successful
"""
return self.network.set_stream_handler(protocol_id, stream_handler)
self.multiselect.add_handler(protocol_id, stream_handler)
# protocol_id can be a list of protocol_ids
# stream will decide which protocol_id to run on
async def new_stream(self, peer_id, protocol_ids):
async def new_stream(
self, peer_id: ID, protocol_ids: Sequence[TProtocol]
) -> INetStream:
"""
:param peer_id: peer_id that host is connecting
:param protocol_id: protocol id that stream runs on
:return: true if successful
:param protocol_ids: available protocol ids to use for stream
:return: stream: new stream created
"""
stream = await self.network.new_stream(peer_id, protocol_ids)
return stream
async def connect(self, peer_info):
net_stream = await self._network.new_stream(peer_id)
# Perform protocol muxing to determine protocol to use
try:
selected_protocol = await self.multiselect_client.select_one_of(
list(protocol_ids), MultiselectCommunicator(net_stream)
)
except MultiselectClientError as error:
logger.debug("fail to open a stream to peer %s, error=%s", peer_id, error)
await net_stream.reset()
raise StreamFailure(f"failed to open a stream to peer {peer_id}") from error
net_stream.set_protocol(selected_protocol)
return net_stream
async def connect(self, peer_info: PeerInfo) -> None:
"""
connect ensures there is a connection between this host and the peer with
given peer_info.peer_id. connect will absorb the addresses in peer_info into its internal
peerstore. If there is not an active connection, connect will issue a
dial, and block until a connection is open, or an error is
returned.
connect ensures there is a connection between this host and the peer
with given `peer_info.peer_id`. connect will absorb the addresses in
peer_info into its internal peerstore. If there is not an active
connection, connect will issue a dial, and block until a connection is
opened, or an error is returned.
:param peer_info: peer_info of the host we want to connect to
:param peer_info: peer_info of the peer we want to connect to
:type peer_info: peer.peerinfo.PeerInfo
"""
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
# there is already a connection to this peer
if peer_info.peer_id in self.network.connections:
if peer_info.peer_id in self._network.connections:
return
await self.network.dial_peer(peer_info.peer_id)
await self._network.dial_peer(peer_info.peer_id)
async def disconnect(self, peer_id: ID) -> None:
await self._network.close_peer(peer_id)
async def close(self) -> None:
await self._network.close()
# Reference: `BasicHost.newStreamHandler` in Go.
async def _swarm_stream_handler(self, net_stream: INetStream) -> None:
# Perform protocol muxing to determine protocol to use
try:
protocol, handler = await self.multiselect.negotiate(
MultiselectCommunicator(net_stream)
)
except MultiselectError as error:
peer_id = net_stream.muxed_conn.peer_id
logger.debug(
"failed to accept a stream from peer %s, error=%s", peer_id, error
)
await net_stream.reset()
return
net_stream.set_protocol(protocol)
await handler(net_stream)

17
libp2p/host/defaults.py Normal file
View File

@ -0,0 +1,17 @@
from collections import OrderedDict
from typing import TYPE_CHECKING
from libp2p.host.host_interface import IHost
from libp2p.host.ping import ID as PingID
from libp2p.host.ping import handle_ping
from libp2p.identity.identify.protocol import ID as IdentifyID
from libp2p.identity.identify.protocol import identify_handler_for
if TYPE_CHECKING:
from libp2p.typing import TProtocol, StreamHandlerFn
def get_default_protocols(host: IHost) -> "OrderedDict[TProtocol, StreamHandlerFn]":
return OrderedDict(
((IdentifyID, identify_handler_for(host)), (PingID, handle_ping))
)

13
libp2p/host/exceptions.py Normal file
View File

@ -0,0 +1,13 @@
from libp2p.exceptions import BaseLibp2pError
class HostException(BaseLibp2pError):
"""A generic exception in `IHost`."""
class ConnectionFailure(HostException):
pass
class StreamFailure(HostException):
pass

View File

@ -1,60 +1,104 @@
from abc import ABC, abstractmethod
from typing import Any, AsyncContextManager, List, Sequence
import multiaddr
from libp2p.crypto.keys import PrivateKey, PublicKey
from libp2p.network.network_interface import INetworkService
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import PeerInfo
from libp2p.typing import StreamHandlerFn, TProtocol
class IHost(ABC):
@abstractmethod
def get_id(self):
def get_id(self) -> ID:
"""
:return: peer_id of host
"""
@abstractmethod
def get_network(self):
def get_public_key(self) -> PublicKey:
"""
:return: the public key belonging to the peer
"""
@abstractmethod
def get_private_key(self) -> PrivateKey:
"""
:return: the private key belonging to the peer
"""
@abstractmethod
def get_network(self) -> INetworkService:
"""
:return: network instance of host
"""
# FIXME: Replace with correct return type
@abstractmethod
def get_mux(self):
def get_mux(self) -> Any:
"""
:return: mux instance of host
"""
@abstractmethod
def get_addrs(self):
def get_addrs(self) -> List[multiaddr.Multiaddr]:
"""
:return: all the multiaddr addresses this host is listening too
:return: all the multiaddr addresses this host is listening to
"""
@abstractmethod
def set_stream_handler(self, protocol_id, stream_handler):
def run(
self, listen_addrs: Sequence[multiaddr.Multiaddr]
) -> AsyncContextManager[None]:
"""
set stream handler for host
run the host instance and listen to ``listen_addrs``.
:param listen_addrs: a sequence of multiaddrs that we want to listen to
"""
@abstractmethod
def set_stream_handler(
self, protocol_id: TProtocol, stream_handler: StreamHandlerFn
) -> None:
"""
set stream handler for host.
:param protocol_id: protocol id used on stream
:param stream_handler: a stream handler function
:return: true if successful
"""
# protocol_id can be a list of protocol_ids
# stream will decide which protocol_id to run on
@abstractmethod
def new_stream(self, peer_id, protocol_ids):
async def new_stream(
self, peer_id: ID, protocol_ids: Sequence[TProtocol]
) -> INetStream:
"""
:param peer_id: peer_id that host is connecting
:param protocol_ids: protocol ids that stream can run on
:return: true if successful
:param protocol_ids: available protocol ids to use for stream
:return: stream: new stream created
"""
@abstractmethod
def connect(self, peer_info):
async def connect(self, peer_info: PeerInfo) -> None:
"""
connect ensures there is a connection between this host and the peer with
given peer_info.peer_id. connect will absorb the addresses in peer_info into its internal
peerstore. If there is not an active connection, connect will issue a
dial, and block until a connection is open, or an error is
returned.
connect ensures there is a connection between this host and the peer
with given peer_info.peer_id. connect will absorb the addresses in
peer_info into its internal peerstore. If there is not an active
connection, connect will issue a dial, and block until a connection is
opened, or an error is returned.
:param peer_info: peer_info of the host we want to connect to
:param peer_info: peer_info of the peer we want to connect to
:type peer_info: peer.peerinfo.PeerInfo
"""
@abstractmethod
async def disconnect(self, peer_id: ID) -> None:
pass
@abstractmethod
async def close(self) -> None:
pass

60
libp2p/host/ping.py Normal file
View File

@ -0,0 +1,60 @@
import logging
import trio
from libp2p.network.stream.exceptions import StreamClosed, StreamEOF, StreamReset
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.peer.id import ID as PeerID
from libp2p.typing import TProtocol
ID = TProtocol("/ipfs/ping/1.0.0")
PING_LENGTH = 32
RESP_TIMEOUT = 60
logger = logging.getLogger("libp2p.host.ping")
async def _handle_ping(stream: INetStream, peer_id: PeerID) -> bool:
"""Return a boolean indicating if we expect more pings from the peer at
``peer_id``."""
try:
with trio.fail_after(RESP_TIMEOUT):
payload = await stream.read(PING_LENGTH)
except trio.TooSlowError as error:
logger.debug("Timed out waiting for ping from %s: %s", peer_id, error)
raise
except StreamEOF:
logger.debug("Other side closed while waiting for ping from %s", peer_id)
return False
except StreamReset as error:
logger.debug(
"Other side reset while waiting for ping from %s: %s", peer_id, error
)
raise
except Exception as error:
logger.debug("Error while waiting to read ping for %s: %s", peer_id, error)
raise
logger.debug("Received ping from %s with data: 0x%s", peer_id, payload.hex())
try:
await stream.write(payload)
except StreamClosed:
logger.debug("Fail to respond to ping from %s: stream closed", peer_id)
raise
return True
async def handle_ping(stream: INetStream) -> None:
"""``handle_ping`` responds to incoming ping requests until one side errors
or closes the ``stream``."""
peer_id = stream.muxed_conn.peer_id
while True:
try:
should_continue = await _handle_ping(stream, peer_id)
if not should_continue:
return
except Exception:
await stream.reset()
return

View File

@ -0,0 +1,41 @@
from libp2p.host.basic_host import BasicHost
from libp2p.host.exceptions import ConnectionFailure
from libp2p.network.network_interface import INetworkService
from libp2p.peer.peerinfo import PeerInfo
from libp2p.routing.interfaces import IPeerRouting
# RoutedHost is a p2p Host that includes a routing system.
# This allows the Host to find the addresses for peers when it does not have them.
class RoutedHost(BasicHost):
_router: IPeerRouting
def __init__(self, network: INetworkService, router: IPeerRouting):
super().__init__(network)
self._router = router
async def connect(self, peer_info: PeerInfo) -> None:
"""
connect ensures there is a connection between this host and the peer
with given `peer_info.peer_id`. See (basic_host).connect for more
information.
RoutedHost's Connect differs in that if the host has no addresses for a
given peer, it will use its routing system to try to find some.
:param peer_info: peer_info of the peer we want to connect to
:type peer_info: peer.peerinfo.PeerInfo
"""
# check if we were given some addresses, otherwise, find some with the routing system.
if not peer_info.addrs:
found_peer_info = await self._router.find_peer(peer_info.peer_id)
if not found_peer_info:
raise ConnectionFailure("Unable to find Peer address")
self.peerstore.add_addrs(peer_info.peer_id, found_peer_info.addrs, 10)
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
# there is already a connection to this peer
if peer_info.peer_id in self._network.connections:
return
await self._network.dial_peer(peer_info.peer_id)

View File

@ -0,0 +1,12 @@
syntax = "proto2";
package identify.pb;
message Identify {
optional string protocol_version = 5;
optional string agent_version = 6;
optional bytes public_key = 1;
repeated bytes listen_addrs = 2;
optional bytes observed_addr = 4;
repeated string protocols = 3;
}

View File

@ -0,0 +1,105 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: libp2p/identity/identify/pb/identify.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='libp2p/identity/identify/pb/identify.proto',
package='identify.pb',
syntax='proto2',
serialized_options=None,
serialized_pb=_b('\n*libp2p/identity/identify/pb/identify.proto\x12\x0bidentify.pb\"\x8f\x01\n\x08Identify\x12\x18\n\x10protocol_version\x18\x05 \x01(\t\x12\x15\n\ragent_version\x18\x06 \x01(\t\x12\x12\n\npublic_key\x18\x01 \x01(\x0c\x12\x14\n\x0clisten_addrs\x18\x02 \x03(\x0c\x12\x15\n\robserved_addr\x18\x04 \x01(\x0c\x12\x11\n\tprotocols\x18\x03 \x03(\t')
)
_IDENTIFY = _descriptor.Descriptor(
name='Identify',
full_name='identify.pb.Identify',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='protocol_version', full_name='identify.pb.Identify.protocol_version', index=0,
number=5, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='agent_version', full_name='identify.pb.Identify.agent_version', index=1,
number=6, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='public_key', full_name='identify.pb.Identify.public_key', index=2,
number=1, type=12, cpp_type=9, label=1,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='listen_addrs', full_name='identify.pb.Identify.listen_addrs', index=3,
number=2, type=12, cpp_type=9, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='observed_addr', full_name='identify.pb.Identify.observed_addr', index=4,
number=4, type=12, cpp_type=9, label=1,
has_default_value=False, default_value=_b(""),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='protocols', full_name='identify.pb.Identify.protocols', index=5,
number=3, type=9, cpp_type=9, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto2',
extension_ranges=[],
oneofs=[
],
serialized_start=60,
serialized_end=203,
)
DESCRIPTOR.message_types_by_name['Identify'] = _IDENTIFY
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
Identify = _reflection.GeneratedProtocolMessageType('Identify', (_message.Message,), {
'DESCRIPTOR' : _IDENTIFY,
'__module__' : 'libp2p.identity.identify.pb.identify_pb2'
# @@protoc_insertion_point(class_scope:identify.pb.Identify)
})
_sym_db.RegisterMessage(Identify)
# @@protoc_insertion_point(module_scope)

View File

@ -0,0 +1,53 @@
# @generated by generate_proto_mypy_stubs.py. Do not edit!
import sys
from google.protobuf.descriptor import (
Descriptor as google___protobuf___descriptor___Descriptor,
)
from google.protobuf.internal.containers import (
RepeatedScalarFieldContainer as google___protobuf___internal___containers___RepeatedScalarFieldContainer,
)
from google.protobuf.message import (
Message as google___protobuf___message___Message,
)
from typing import (
Iterable as typing___Iterable,
Optional as typing___Optional,
Text as typing___Text,
)
from typing_extensions import (
Literal as typing_extensions___Literal,
)
class Identify(google___protobuf___message___Message):
DESCRIPTOR: google___protobuf___descriptor___Descriptor = ...
protocol_version = ... # type: typing___Text
agent_version = ... # type: typing___Text
public_key = ... # type: bytes
listen_addrs = ... # type: google___protobuf___internal___containers___RepeatedScalarFieldContainer[bytes]
observed_addr = ... # type: bytes
protocols = ... # type: google___protobuf___internal___containers___RepeatedScalarFieldContainer[typing___Text]
def __init__(self,
*,
protocol_version : typing___Optional[typing___Text] = None,
agent_version : typing___Optional[typing___Text] = None,
public_key : typing___Optional[bytes] = None,
listen_addrs : typing___Optional[typing___Iterable[bytes]] = None,
observed_addr : typing___Optional[bytes] = None,
protocols : typing___Optional[typing___Iterable[typing___Text]] = None,
) -> None: ...
@classmethod
def FromString(cls, s: bytes) -> Identify: ...
def MergeFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
def CopyFrom(self, other_msg: google___protobuf___message___Message) -> None: ...
if sys.version_info >= (3,):
def HasField(self, field_name: typing_extensions___Literal[u"agent_version",u"observed_addr",u"protocol_version",u"public_key"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"agent_version",u"listen_addrs",u"observed_addr",u"protocol_version",u"protocols",u"public_key"]) -> None: ...
else:
def HasField(self, field_name: typing_extensions___Literal[u"agent_version",b"agent_version",u"observed_addr",b"observed_addr",u"protocol_version",b"protocol_version",u"public_key",b"public_key"]) -> bool: ...
def ClearField(self, field_name: typing_extensions___Literal[u"agent_version",b"agent_version",u"listen_addrs",b"listen_addrs",u"observed_addr",b"observed_addr",u"protocol_version",b"protocol_version",u"protocols",b"protocols",u"public_key",b"public_key"]) -> None: ...

View File

@ -0,0 +1,55 @@
import logging
from multiaddr import Multiaddr
from libp2p.host.host_interface import IHost
from libp2p.network.stream.exceptions import StreamClosed
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.typing import StreamHandlerFn, TProtocol
from .pb.identify_pb2 import Identify
ID = TProtocol("/ipfs/id/1.0.0")
PROTOCOL_VERSION = "ipfs/0.1.0"
# TODO dynamically generate the agent version
AGENT_VERSION = "py-libp2p/alpha"
logger = logging.getLogger("libp2p.identity.identify")
def _multiaddr_to_bytes(maddr: Multiaddr) -> bytes:
return maddr.to_bytes()
def _mk_identify_protobuf(host: IHost) -> Identify:
public_key = host.get_public_key()
laddrs = host.get_addrs()
protocols = host.get_mux().get_protocols()
return Identify(
protocol_version=PROTOCOL_VERSION,
agent_version=AGENT_VERSION,
public_key=public_key.serialize(),
listen_addrs=map(_multiaddr_to_bytes, laddrs),
# TODO send observed address from ``stream``
observed_addr=b"",
protocols=protocols,
)
def identify_handler_for(host: IHost) -> StreamHandlerFn:
async def handle_identify(stream: INetStream) -> None:
peer_id = stream.muxed_conn.peer_id
logger.debug("received a request for %s from %s", ID, peer_id)
protobuf = _mk_identify_protobuf(host)
response = protobuf.SerializeToString()
try:
await stream.write(response)
except StreamClosed:
logger.debug("Fail to respond to %s request: stream closed", ID)
else:
await stream.close()
logger.debug("successfully handled request for %s from %s", ID, peer_id)
return handle_identify

0
libp2p/io/__init__.py Normal file
View File

65
libp2p/io/abc.py Normal file
View File

@ -0,0 +1,65 @@
from abc import ABC, abstractmethod
class Closer(ABC):
@abstractmethod
async def close(self) -> None:
...
class Reader(ABC):
@abstractmethod
async def read(self, n: int = None) -> bytes:
...
class Writer(ABC):
@abstractmethod
async def write(self, data: bytes) -> None:
...
class WriteCloser(Writer, Closer):
pass
class ReadCloser(Reader, Closer):
pass
class ReadWriter(Reader, Writer):
pass
class ReadWriteCloser(Reader, Writer, Closer):
pass
class MsgReader(ABC):
@abstractmethod
async def read_msg(self) -> bytes:
...
class MsgWriter(ABC):
@abstractmethod
async def write_msg(self, msg: bytes) -> None:
...
class MsgReadWriteCloser(MsgReader, MsgWriter, Closer):
pass
class Encrypter(ABC):
@abstractmethod
def encrypt(self, data: bytes) -> bytes:
...
@abstractmethod
def decrypt(self, data: bytes) -> bytes:
...
class EncryptedMsgReadWriter(MsgReadWriteCloser, Encrypter):
"""Read/write message with encryption/decryption."""

29
libp2p/io/exceptions.py Normal file
View File

@ -0,0 +1,29 @@
from libp2p.exceptions import BaseLibp2pError
class IOException(BaseLibp2pError):
pass
class IncompleteReadError(IOException):
"""Fewer bytes were read than requested."""
class MsgioException(IOException):
pass
class MissingLengthException(MsgioException):
pass
class MissingMessageException(MsgioException):
pass
class DecryptionFailedException(MsgioException):
pass
class MessageTooLarge(MsgioException):
pass

89
libp2p/io/msgio.py Normal file
View File

@ -0,0 +1,89 @@
"""
``msgio`` is an implementation of `https://github.com/libp2p/go-msgio`.
from that repo: "a simple package to r/w length-delimited slices."
NOTE: currently missing the capability to indicate lengths by "varint" method.
"""
from abc import abstractmethod
from libp2p.io.abc import MsgReadWriteCloser, Reader, ReadWriteCloser
from libp2p.io.utils import read_exactly
from libp2p.utils import decode_uvarint_from_stream, encode_varint_prefixed
from .exceptions import MessageTooLarge
BYTE_ORDER = "big"
async def read_length(reader: Reader, size_len_bytes: int) -> int:
length_bytes = await read_exactly(reader, size_len_bytes)
return int.from_bytes(length_bytes, byteorder=BYTE_ORDER)
def encode_msg_with_length(msg_bytes: bytes, size_len_bytes: int) -> bytes:
try:
len_prefix = len(msg_bytes).to_bytes(size_len_bytes, byteorder=BYTE_ORDER)
except OverflowError:
raise ValueError(
"msg_bytes is too large for `size_len_bytes` bytes length: "
f"msg_bytes={msg_bytes!r}, size_len_bytes={size_len_bytes}"
)
return len_prefix + msg_bytes
class BaseMsgReadWriter(MsgReadWriteCloser):
read_write_closer: ReadWriteCloser
size_len_bytes: int
def __init__(self, read_write_closer: ReadWriteCloser) -> None:
self.read_write_closer = read_write_closer
async def read_msg(self) -> bytes:
length = await self.next_msg_len()
return await read_exactly(self.read_write_closer, length)
@abstractmethod
async def next_msg_len(self) -> int:
...
@abstractmethod
def encode_msg(self, msg: bytes) -> bytes:
...
async def close(self) -> None:
await self.read_write_closer.close()
async def write_msg(self, msg: bytes) -> None:
encoded_msg = self.encode_msg(msg)
await self.read_write_closer.write(encoded_msg)
class FixedSizeLenMsgReadWriter(BaseMsgReadWriter):
size_len_bytes: int
async def next_msg_len(self) -> int:
return await read_length(self.read_write_closer, self.size_len_bytes)
def encode_msg(self, msg: bytes) -> bytes:
return encode_msg_with_length(msg, self.size_len_bytes)
class VarIntLengthMsgReadWriter(BaseMsgReadWriter):
max_msg_size: int
async def next_msg_len(self) -> int:
msg_len = await decode_uvarint_from_stream(self.read_write_closer)
if msg_len > self.max_msg_size:
raise MessageTooLarge(
f"msg_len={msg_len} > max_msg_size={self.max_msg_size}"
)
return msg_len
def encode_msg(self, msg: bytes) -> bytes:
msg_len = len(msg)
if msg_len > self.max_msg_size:
raise MessageTooLarge(
f"msg_len={msg_len} > max_msg_size={self.max_msg_size}"
)
return encode_varint_prefixed(msg)

40
libp2p/io/trio.py Normal file
View File

@ -0,0 +1,40 @@
import logging
import trio
from libp2p.io.abc import ReadWriteCloser
from libp2p.io.exceptions import IOException
logger = logging.getLogger("libp2p.io.trio")
class TrioTCPStream(ReadWriteCloser):
stream: trio.SocketStream
# NOTE: Add both read and write lock to avoid `trio.BusyResourceError`
read_lock: trio.Lock
write_lock: trio.Lock
def __init__(self, stream: trio.SocketStream) -> None:
self.stream = stream
self.read_lock = trio.Lock()
self.write_lock = trio.Lock()
async def write(self, data: bytes) -> None:
"""Raise `RawConnError` if the underlying connection breaks."""
async with self.write_lock:
try:
await self.stream.send_all(data)
except (trio.ClosedResourceError, trio.BrokenResourceError) as error:
raise IOException from error
async def read(self, n: int = None) -> bytes:
async with self.read_lock:
if n is not None and n == 0:
return b""
try:
return await self.stream.receive_some(n)
except (trio.ClosedResourceError, trio.BrokenResourceError) as error:
raise IOException from error
async def close(self) -> None:
await self.stream.aclose()

21
libp2p/io/utils.py Normal file
View File

@ -0,0 +1,21 @@
from libp2p.io.abc import Reader
from libp2p.io.exceptions import IncompleteReadError
DEFAULT_RETRY_READ_COUNT = 100
async def read_exactly(
reader: Reader, n: int, retry_count: int = DEFAULT_RETRY_READ_COUNT
) -> bytes:
"""
NOTE: relying on exceptions to break out on erroneous conditions, like EOF
"""
data = await reader.read(n)
for _ in range(retry_count):
if len(data) < n:
remaining = n - len(data)
data += await reader.read(remaining)
else:
return data
raise IncompleteReadError({"requested_count": n, "received_count": len(data)})

View File

@ -1,5 +0,0 @@
"""
Kademlia is a Python implementation of the Kademlia protocol which
utilizes the asyncio library.
"""
__version__ = "1.1"

View File

@ -1,181 +0,0 @@
from collections import Counter
import logging
from .kademlia.node import Node, NodeHeap
from .kademlia.utils import gather_dict
log = logging.getLogger(__name__)
class SpiderCrawl:
"""
Crawl the network and look for given 160-bit keys.
"""
def __init__(self, protocol, node, peers, ksize, alpha):
"""
Create a new C{SpiderCrawl}er.
Args:
protocol: A :class:`~kademlia.protocol.KademliaProtocol` instance.
node: A :class:`~kademlia.node.Node` representing the key we're
looking for
peers: A list of :class:`~kademlia.node.Node` instances that
provide the entry point for the network
ksize: The value for k based on the paper
alpha: The value for alpha based on the paper
"""
self.protocol = protocol
self.ksize = ksize
self.alpha = alpha
self.node = node
self.nearest = NodeHeap(self.node, self.ksize)
self.lastIDsCrawled = []
log.info("creating spider with peers: %s", peers)
self.nearest.push(peers)
async def _find(self, rpcmethod):
"""
Get either a value or list of nodes.
Args:
rpcmethod: The protocol's callfindValue or callFindNode.
The process:
1. calls find_* to current ALPHA nearest not already queried nodes,
adding results to current nearest list of k nodes.
2. current nearest list needs to keep track of who has been queried
already sort by nearest, keep KSIZE
3. if list is same as last time, next call should be to everyone not
yet queried
4. repeat, unless nearest list has all been queried, then ur done
"""
log.info("crawling network with nearest: %s", str(tuple(self.nearest)))
count = self.alpha
if self.nearest.getIDs() == self.lastIDsCrawled:
count = len(self.nearest)
self.lastIDsCrawled = self.nearest.getIDs()
ds = {}
for peer in self.nearest.getUncontacted()[:count]:
ds[peer.id] = rpcmethod(peer, self.node)
self.nearest.markContacted(peer)
found = await gather_dict(ds)
return await self._nodesFound(found)
async def _nodesFound(self, responses):
raise NotImplementedError
class ValueSpiderCrawl(SpiderCrawl):
def __init__(self, protocol, node, peers, ksize, alpha):
SpiderCrawl.__init__(self, protocol, node, peers, ksize, alpha)
# keep track of the single nearest node without value - per
# section 2.3 so we can set the key there if found
self.nearestWithoutValue = NodeHeap(self.node, 1)
async def find(self):
"""
Find either the closest nodes or the value requested.
"""
return await self._find(self.protocol.callFindValue)
async def _nodesFound(self, responses):
"""
Handle the result of an iteration in _find.
"""
toremove = []
foundValues = []
for peerid, response in responses.items():
response = RPCFindResponse(response)
if not response.happened():
toremove.append(peerid)
elif response.hasValue():
foundValues.append(response.getValue())
else:
peer = self.nearest.getNodeById(peerid)
self.nearestWithoutValue.push(peer)
self.nearest.push(response.getNodeList())
self.nearest.remove(toremove)
if len(foundValues) > 0:
return await self._handleFoundValues(foundValues)
if self.nearest.allBeenContacted():
# not found!
return None
return await self.find()
async def _handleFoundValues(self, values):
"""
We got some values! Exciting. But let's make sure
they're all the same or freak out a little bit. Also,
make sure we tell the nearest node that *didn't* have
the value to store it.
"""
valueCounts = Counter(values)
if len(valueCounts) != 1:
log.warning("Got multiple values for key %i: %s",
self.node.long_id, str(values))
value = valueCounts.most_common(1)[0][0]
peerToSaveTo = self.nearestWithoutValue.popleft()
if peerToSaveTo is not None:
await self.protocol.callStore(peerToSaveTo, self.node.id, value)
return value
class NodeSpiderCrawl(SpiderCrawl):
async def find(self):
"""
Find the closest nodes.
"""
return await self._find(self.protocol.callFindNode)
async def _nodesFound(self, responses):
"""
Handle the result of an iteration in _find.
"""
toremove = []
for peerid, response in responses.items():
response = RPCFindResponse(response)
if not response.happened():
toremove.append(peerid)
else:
self.nearest.push(response.getNodeList())
self.nearest.remove(toremove)
if self.nearest.allBeenContacted():
return list(self.nearest)
return await self.find()
class RPCFindResponse:
def __init__(self, response):
"""
A wrapper for the result of a RPC find.
Args:
response: This will be a tuple of (<response received>, <value>)
where <value> will be a list of tuples if not found or
a dictionary of {'value': v} where v is the value desired
"""
self.response = response
def happened(self):
"""
Did the other host actually respond?
"""
return self.response[0]
def hasValue(self):
return isinstance(self.response[1], dict)
def getValue(self):
return self.response[1]['value']
def getNodeList(self):
"""
Get the node list in the response. If there's no value, this should
be set.
"""
nodelist = self.response[1] or []
return [Node(*nodeple) for nodeple in nodelist]

View File

@ -1,258 +0,0 @@
"""
Package for interacting on the network at a high level.
"""
import random
import pickle
import asyncio
import logging
from .kademlia.protocol import KademliaProtocol
from .kademlia.utils import digest
from .kademlia.storage import ForgetfulStorage
from .kademlia.node import Node
from .kademlia.crawling import ValueSpiderCrawl
from .kademlia.crawling import NodeSpiderCrawl
log = logging.getLogger(__name__)
class Server:
"""
High level view of a node instance. This is the object that should be
created to start listening as an active node on the network.
"""
protocol_class = KademliaProtocol
def __init__(self, ksize=20, alpha=3, node_id=None, storage=None):
"""
Create a server instance. This will start listening on the given port.
Args:
ksize (int): The k parameter from the paper
alpha (int): The alpha parameter from the paper
node_id: The id for this node on the network.
storage: An instance that implements
:interface:`~kademlia.storage.IStorage`
"""
self.ksize = ksize
self.alpha = alpha
self.storage = storage or ForgetfulStorage()
self.node = Node(node_id or digest(random.getrandbits(255)))
self.transport = None
self.protocol = None
self.refresh_loop = None
self.save_state_loop = None
def stop(self):
if self.transport is not None:
self.transport.close()
if self.refresh_loop:
self.refresh_loop.cancel()
if self.save_state_loop:
self.save_state_loop.cancel()
def _create_protocol(self):
return self.protocol_class(self.node, self.storage, self.ksize)
def listen(self, port, interface='0.0.0.0'):
"""
Start listening on the given port.
Provide interface="::" to accept ipv6 address
"""
loop = asyncio.get_event_loop()
listen = loop.create_datagram_endpoint(self._create_protocol,
local_addr=(interface, port))
log.info("Node %i listening on %s:%i",
self.node.long_id, interface, port)
self.transport, self.protocol = loop.run_until_complete(listen)
# finally, schedule refreshing table
self.refresh_table()
def refresh_table(self):
log.debug("Refreshing routing table")
asyncio.ensure_future(self._refresh_table())
loop = asyncio.get_event_loop()
self.refresh_loop = loop.call_later(3600, self.refresh_table)
async def _refresh_table(self):
"""
Refresh buckets that haven't had any lookups in the last hour
(per section 2.3 of the paper).
"""
ds = []
for node_id in self.protocol.getRefreshIDs():
node = Node(node_id)
nearest = self.protocol.router.findNeighbors(node, self.alpha)
spider = NodeSpiderCrawl(self.protocol, node, nearest,
self.ksize, self.alpha)
ds.append(spider.find())
# do our crawling
await asyncio.gather(*ds)
# now republish keys older than one hour
for dkey, value in self.storage.iteritemsOlderThan(3600):
await self.set_digest(dkey, value)
def bootstrappableNeighbors(self):
"""
Get a :class:`list` of (ip, port) :class:`tuple` pairs suitable for
use as an argument to the bootstrap method.
The server should have been bootstrapped
already - this is just a utility for getting some neighbors and then
storing them if this server is going down for a while. When it comes
back up, the list of nodes can be used to bootstrap.
"""
neighbors = self.protocol.router.findNeighbors(self.node)
return [tuple(n)[-2:] for n in neighbors]
async def bootstrap(self, addrs):
"""
Bootstrap the server by connecting to other known nodes in the network.
Args:
addrs: A `list` of (ip, port) `tuple` pairs. Note that only IP
addresses are acceptable - hostnames will cause an error.
"""
log.debug("Attempting to bootstrap node with %i initial contacts",
len(addrs))
cos = list(map(self.bootstrap_node, addrs))
gathered = await asyncio.gather(*cos)
nodes = [node for node in gathered if node is not None]
spider = NodeSpiderCrawl(self.protocol, self.node, nodes,
self.ksize, self.alpha)
return await spider.find()
async def bootstrap_node(self, addr):
result = await self.protocol.ping(addr, self.node.id)
return Node(result[1], addr[0], addr[1]) if result[0] else None
async def get(self, key):
"""
Get a key if the network has it.
Returns:
:class:`None` if not found, the value otherwise.
"""
log.info("Looking up key %s", key)
dkey = digest(key)
# if this node has it, return it
if self.storage.get(dkey) is not None:
return self.storage.get(dkey)
node = Node(dkey)
nearest = self.protocol.router.findNeighbors(node)
if len(nearest) == 0:
log.warning("There are no known neighbors to get key %s", key)
return None
spider = ValueSpiderCrawl(self.protocol, node, nearest,
self.ksize, self.alpha)
return await spider.find()
async def set(self, key, value):
"""
Set the given string key to the given value in the network.
"""
if not check_dht_value_type(value):
raise TypeError(
"Value must be of type int, float, bool, str, or bytes"
)
log.info("setting '%s' = '%s' on network", key, value)
dkey = digest(key)
return await self.set_digest(dkey, value)
async def set_digest(self, dkey, value):
"""
Set the given SHA1 digest key (bytes) to the given value in the
network.
"""
node = Node(dkey)
nearest = self.protocol.router.findNeighbors(node)
if len(nearest) == 0:
log.warning("There are no known neighbors to set key %s",
dkey.hex())
return False
spider = NodeSpiderCrawl(self.protocol, node, nearest,
self.ksize, self.alpha)
nodes = await spider.find()
log.info("setting '%s' on %s", dkey.hex(), list(map(str, nodes)))
# if this node is close too, then store here as well
biggest = max([n.distanceTo(node) for n in nodes])
if self.node.distanceTo(node) < biggest:
self.storage[dkey] = value
ds = [self.protocol.callStore(n, dkey, value) for n in nodes]
# return true only if at least one store call succeeded
return any(await asyncio.gather(*ds))
def saveState(self, fname):
"""
Save the state of this node (the alpha/ksize/id/immediate neighbors)
to a cache file with the given fname.
"""
log.info("Saving state to %s", fname)
data = {
'ksize': self.ksize,
'alpha': self.alpha,
'id': self.node.id,
'neighbors': self.bootstrappableNeighbors()
}
if len(data['neighbors']) == 0:
log.warning("No known neighbors, so not writing to cache.")
return
with open(fname, 'wb') as f:
pickle.dump(data, f)
@classmethod
def loadState(self, fname):
"""
Load the state of this node (the alpha/ksize/id/immediate neighbors)
from a cache file with the given fname.
"""
log.info("Loading state from %s", fname)
with open(fname, 'rb') as f:
data = pickle.load(f)
s = Server(data['ksize'], data['alpha'], data['id'])
if len(data['neighbors']) > 0:
s.bootstrap(data['neighbors'])
return s
def saveStateRegularly(self, fname, frequency=600):
"""
Save the state of node with a given regularity to the given
filename.
Args:
fname: File name to save retularly to
frequency: Frequency in seconds that the state should be saved.
By default, 10 minutes.
"""
self.saveState(fname)
loop = asyncio.get_event_loop()
self.save_state_loop = loop.call_later(frequency,
self.saveStateRegularly,
fname,
frequency)
def check_dht_value_type(value):
"""
Checks to see if the type of the value is a valid type for
placing in the dht.
"""
typeset = set(
[
int,
float,
bool,
str,
bytes,
]
)
return type(value) in typeset

View File

@ -1,115 +0,0 @@
from operator import itemgetter
import heapq
class Node:
def __init__(self, node_id, ip=None, port=None):
self.id = node_id
self.ip = ip
self.port = port
self.long_id = int(node_id.hex(), 16)
def sameHomeAs(self, node):
return self.ip == node.ip and self.port == node.port
def distanceTo(self, node):
"""
Get the distance between this node and another.
"""
return self.long_id ^ node.long_id
def __iter__(self):
"""
Enables use of Node as a tuple - i.e., tuple(node) works.
"""
return iter([self.id, self.ip, self.port])
def __repr__(self):
return repr([self.long_id, self.ip, self.port])
def __str__(self):
return "%s:%s" % (self.ip, str(self.port))
class NodeHeap:
"""
A heap of nodes ordered by distance to a given node.
"""
def __init__(self, node, maxsize):
"""
Constructor.
@param node: The node to measure all distnaces from.
@param maxsize: The maximum size that this heap can grow to.
"""
self.node = node
self.heap = []
self.contacted = set()
self.maxsize = maxsize
def remove(self, peerIDs):
"""
Remove a list of peer ids from this heap. Note that while this
heap retains a constant visible size (based on the iterator), it's
actual size may be quite a bit larger than what's exposed. Therefore,
removal of nodes may not change the visible size as previously added
nodes suddenly become visible.
"""
peerIDs = set(peerIDs)
if len(peerIDs) == 0:
return
nheap = []
for distance, node in self.heap:
if node.id not in peerIDs:
heapq.heappush(nheap, (distance, node))
self.heap = nheap
def getNodeById(self, node_id):
for _, node in self.heap:
if node.id == node_id:
return node
return None
def allBeenContacted(self):
return len(self.getUncontacted()) == 0
def getIDs(self):
return [n.id for n in self]
def markContacted(self, node):
self.contacted.add(node.id)
def popleft(self):
if len(self) > 0:
return heapq.heappop(self.heap)[1]
return None
def push(self, nodes):
"""
Push nodes onto heap.
@param nodes: This can be a single item or a C{list}.
"""
if not isinstance(nodes, list):
nodes = [nodes]
for node in nodes:
if node not in self:
distance = self.node.distanceTo(node)
heapq.heappush(self.heap, (distance, node))
def __len__(self):
return min(len(self.heap), self.maxsize)
def __iter__(self):
nodes = heapq.nsmallest(self.maxsize, self.heap)
return iter(map(itemgetter(1), nodes))
def __contains__(self, node):
for _, n in self.heap:
if node.id == n.id:
return True
return False
def getUncontacted(self):
return [n for n in self if n.id not in self.contacted]

View File

@ -1,128 +0,0 @@
import random
import asyncio
import logging
from rpcudp.protocol import RPCProtocol
from .kademlia.node import Node
from .kademlia.routing import RoutingTable
from .kademlia.utils import digest
log = logging.getLogger(__name__)
class KademliaProtocol(RPCProtocol):
def __init__(self, sourceNode, storage, ksize):
RPCProtocol.__init__(self)
self.router = RoutingTable(self, ksize, sourceNode)
self.storage = storage
self.sourceNode = sourceNode
def getRefreshIDs(self):
"""
Get ids to search for to keep old buckets up to date.
"""
ids = []
for bucket in self.router.getLonelyBuckets():
rid = random.randint(*bucket.range).to_bytes(20, byteorder='big')
ids.append(rid)
return ids
def rpc_stun(self, sender):
return sender
def rpc_ping(self, sender, nodeid):
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
return self.sourceNode.id
def rpc_store(self, sender, nodeid, key, value):
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
log.debug("got a store request from %s, storing '%s'='%s'",
sender, key.hex(), value)
self.storage[key] = value
return True
def rpc_find_node(self, sender, nodeid, key):
log.info("finding neighbors of %i in local table",
int(nodeid.hex(), 16))
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
node = Node(key)
neighbors = self.router.findNeighbors(node, exclude=source)
return list(map(tuple, neighbors))
def rpc_find_value(self, sender, nodeid, key):
source = Node(nodeid, sender[0], sender[1])
self.welcomeIfNewNode(source)
value = self.storage.get(key, None)
if value is None:
return self.rpc_find_node(sender, nodeid, key)
return {'value': value}
async def callFindNode(self, nodeToAsk, nodeToFind):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.find_node(address, self.sourceNode.id,
nodeToFind.id)
return self.handleCallResponse(result, nodeToAsk)
async def callFindValue(self, nodeToAsk, nodeToFind):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.find_value(address, self.sourceNode.id,
nodeToFind.id)
return self.handleCallResponse(result, nodeToAsk)
async def callPing(self, nodeToAsk):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.ping(address, self.sourceNode.id)
return self.handleCallResponse(result, nodeToAsk)
async def callStore(self, nodeToAsk, key, value):
address = (nodeToAsk.ip, nodeToAsk.port)
result = await self.store(address, self.sourceNode.id, key, value)
return self.handleCallResponse(result, nodeToAsk)
def welcomeIfNewNode(self, node):
"""
Given a new node, send it all the keys/values it should be storing,
then add it to the routing table.
@param node: A new node that just joined (or that we just found out
about).
Process:
For each key in storage, get k closest nodes. If newnode is closer
than the furtherst in that list, and the node for this server
is closer than the closest in that list, then store the key/value
on the new node (per section 2.5 of the paper)
"""
if not self.router.isNewNode(node):
return
log.info("never seen %s before, adding to router", node)
for key, value in self.storage.items():
keynode = Node(digest(key))
neighbors = self.router.findNeighbors(keynode)
if len(neighbors) > 0:
last = neighbors[-1].distanceTo(keynode)
newNodeClose = node.distanceTo(keynode) < last
first = neighbors[0].distanceTo(keynode)
thisNodeClosest = self.sourceNode.distanceTo(keynode) < first
if len(neighbors) == 0 or (newNodeClose and thisNodeClosest):
asyncio.ensure_future(self.callStore(node, key, value))
self.router.addContact(node)
def handleCallResponse(self, result, node):
"""
If we get a response, add the node to the routing table. If
we get no response, make sure it's removed from the routing table.
"""
if not result[0]:
log.warning("no response from %s, removing from router", node)
self.router.removeContact(node)
return result
log.info("got successful response from %s", node)
self.welcomeIfNewNode(node)
return result

View File

@ -1,185 +0,0 @@
import heapq
import time
import operator
import asyncio
from collections import OrderedDict
from .kademlia.utils import OrderedSet, sharedPrefix, bytesToBitString
class KBucket:
def __init__(self, rangeLower, rangeUpper, ksize):
self.range = (rangeLower, rangeUpper)
self.nodes = OrderedDict()
self.replacementNodes = OrderedSet()
self.touchLastUpdated()
self.ksize = ksize
def touchLastUpdated(self):
self.lastUpdated = time.monotonic()
def getNodes(self):
return list(self.nodes.values())
def split(self):
midpoint = (self.range[0] + self.range[1]) / 2
one = KBucket(self.range[0], midpoint, self.ksize)
two = KBucket(midpoint + 1, self.range[1], self.ksize)
for node in self.nodes.values():
bucket = one if node.long_id <= midpoint else two
bucket.nodes[node.id] = node
return (one, two)
def removeNode(self, node):
if node.id not in self.nodes:
return
# delete node, and see if we can add a replacement
del self.nodes[node.id]
if len(self.replacementNodes) > 0:
newnode = self.replacementNodes.pop()
self.nodes[newnode.id] = newnode
def hasInRange(self, node):
return self.range[0] <= node.long_id <= self.range[1]
def isNewNode(self, node):
return node.id not in self.nodes
def addNode(self, node):
"""
Add a C{Node} to the C{KBucket}. Return True if successful,
False if the bucket is full.
If the bucket is full, keep track of node in a replacement list,
per section 4.1 of the paper.
"""
if node.id in self.nodes:
del self.nodes[node.id]
self.nodes[node.id] = node
elif len(self) < self.ksize:
self.nodes[node.id] = node
else:
self.replacementNodes.push(node)
return False
return True
def depth(self):
vals = self.nodes.values()
sp = sharedPrefix([bytesToBitString(n.id) for n in vals])
return len(sp)
def head(self):
return list(self.nodes.values())[0]
def __getitem__(self, node_id):
return self.nodes.get(node_id, None)
def __len__(self):
return len(self.nodes)
class TableTraverser:
def __init__(self, table, startNode):
index = table.getBucketFor(startNode)
table.buckets[index].touchLastUpdated()
self.currentNodes = table.buckets[index].getNodes()
self.leftBuckets = table.buckets[:index]
self.rightBuckets = table.buckets[(index + 1):]
self.left = True
def __iter__(self):
return self
def __next__(self):
"""
Pop an item from the left subtree, then right, then left, etc.
"""
if len(self.currentNodes) > 0:
return self.currentNodes.pop()
if self.left and len(self.leftBuckets) > 0:
self.currentNodes = self.leftBuckets.pop().getNodes()
self.left = False
return next(self)
if len(self.rightBuckets) > 0:
self.currentNodes = self.rightBuckets.pop(0).getNodes()
self.left = True
return next(self)
raise StopIteration
class RoutingTable:
def __init__(self, protocol, ksize, node):
"""
@param node: The node that represents this server. It won't
be added to the routing table, but will be needed later to
determine which buckets to split or not.
"""
self.node = node
self.protocol = protocol
self.ksize = ksize
self.flush()
def flush(self):
self.buckets = [KBucket(0, 2 ** 160, self.ksize)]
def splitBucket(self, index):
one, two = self.buckets[index].split()
self.buckets[index] = one
self.buckets.insert(index + 1, two)
def getLonelyBuckets(self):
"""
Get all of the buckets that haven't been updated in over
an hour.
"""
hrago = time.monotonic() - 3600
return [b for b in self.buckets if b.lastUpdated < hrago]
def removeContact(self, node):
index = self.getBucketFor(node)
self.buckets[index].removeNode(node)
def isNewNode(self, node):
index = self.getBucketFor(node)
return self.buckets[index].isNewNode(node)
def addContact(self, node):
index = self.getBucketFor(node)
bucket = self.buckets[index]
# this will succeed unless the bucket is full
if bucket.addNode(node):
return
# Per section 4.2 of paper, split if the bucket has the node
# in its range or if the depth is not congruent to 0 mod 5
if bucket.hasInRange(self.node) or bucket.depth() % 5 != 0:
self.splitBucket(index)
self.addContact(node)
else:
asyncio.ensure_future(self.protocol.callPing(bucket.head()))
def getBucketFor(self, node):
"""
Get the index of the bucket that the given node would fall into.
"""
for index, bucket in enumerate(self.buckets):
if node.long_id < bucket.range[1]:
return index
def findNeighbors(self, node, k=None, exclude=None):
k = k or self.ksize
nodes = []
for neighbor in TableTraverser(self, node):
notexcluded = exclude is None or not neighbor.sameHomeAs(exclude)
if neighbor.id != node.id and notexcluded:
heapq.heappush(nodes, (node.distanceTo(neighbor), neighbor))
if len(nodes) == k:
break
return list(map(operator.itemgetter(1), heapq.nsmallest(k, nodes)))

View File

@ -1,97 +0,0 @@
import time
from itertools import takewhile
import operator
from collections import OrderedDict
class IStorage:
"""
Local storage for this node.
IStorage implementations of get must return the same type as put in by set
"""
def __setitem__(self, key, value):
"""
Set a key to the given value.
"""
raise NotImplementedError
def __getitem__(self, key):
"""
Get the given key. If item doesn't exist, raises C{KeyError}
"""
raise NotImplementedError
def get(self, key, default=None):
"""
Get given key. If not found, return default.
"""
raise NotImplementedError
def iteritemsOlderThan(self, secondsOld):
"""
Return the an iterator over (key, value) tuples for items older
than the given secondsOld.
"""
raise NotImplementedError
def __iter__(self):
"""
Get the iterator for this storage, should yield tuple of (key, value)
"""
raise NotImplementedError
class ForgetfulStorage(IStorage):
def __init__(self, ttl=604800):
"""
By default, max age is a week.
"""
self.data = OrderedDict()
self.ttl = ttl
def __setitem__(self, key, value):
if key in self.data:
del self.data[key]
self.data[key] = (time.monotonic(), value)
self.cull()
def cull(self):
for _, _ in self.iteritemsOlderThan(self.ttl):
self.data.popitem(last=False)
def get(self, key, default=None):
self.cull()
if key in self.data:
return self[key]
return default
def __getitem__(self, key):
self.cull()
return self.data[key][1]
def __iter__(self):
self.cull()
return iter(self.data)
def __repr__(self):
self.cull()
return repr(self.data)
def iteritemsOlderThan(self, secondsOld):
minBirthday = time.monotonic() - secondsOld
zipped = self._tripleIterable()
matches = takewhile(lambda r: minBirthday >= r[1], zipped)
return list(map(operator.itemgetter(0, 2), matches))
def _tripleIterable(self):
ikeys = self.data.keys()
ibirthday = map(operator.itemgetter(0), self.data.values())
ivalues = map(operator.itemgetter(1), self.data.values())
return zip(ikeys, ibirthday, ivalues)
def items(self):
self.cull()
ikeys = self.data.keys()
ivalues = map(operator.itemgetter(1), self.data.values())
return zip(ikeys, ivalues)

View File

@ -1,57 +0,0 @@
"""
General catchall for functions that don't make sense as methods.
"""
import hashlib
import operator
import asyncio
async def gather_dict(d):
cors = list(d.values())
results = await asyncio.gather(*cors)
return dict(zip(d.keys(), results))
def digest(s):
if not isinstance(s, bytes):
s = str(s).encode('utf8')
return hashlib.sha1(s).digest()
class OrderedSet(list):
"""
Acts like a list in all ways, except in the behavior of the
:meth:`push` method.
"""
def push(self, thing):
"""
1. If the item exists in the list, it's removed
2. The item is pushed to the end of the list
"""
if thing in self:
self.remove(thing)
self.append(thing)
def sharedPrefix(args):
"""
Find the shared prefix between the strings.
For instance:
sharedPrefix(['blahblah', 'blahwhat'])
returns 'blah'.
"""
i = 0
while i < min(map(len, args)):
if len(set(map(operator.itemgetter(i), args))) != 1:
break
i += 1
return args[0][:i]
def bytesToBitString(bites):
bits = [bin(bite)[2:].rjust(8, '0') for bite in bites]
return "".join(bits)

View File

@ -0,0 +1,5 @@
from libp2p.io.exceptions import IOException
class RawConnError(IOException):
pass

View File

@ -0,0 +1,21 @@
from abc import abstractmethod
from typing import Tuple
import trio
from libp2p.io.abc import Closer
from libp2p.network.stream.net_stream_interface import INetStream
from libp2p.stream_muxer.abc import IMuxedConn
class INetConn(Closer):
muxed_conn: IMuxedConn
event_started: trio.Event
@abstractmethod
async def new_stream(self) -> INetStream:
...
@abstractmethod
def get_streams(self) -> Tuple[INetStream, ...]:
...

Some files were not shown because too many files have changed in this diff Show More