StarMirror/toxcore
1
0
Fork 0
mirror of https://github.com/irungentoo/toxcore.git synced 2024-03-22 13:30:51 +08:00
Code Issues Projects Releases Wiki Activity

Merge pull request #71 from ollieh/master

Browse Source 
Removed docs, all are now in the wiki
This commit is contained in:
irungentoo 2013-07-21 06:02:37 -07:00
commit 71fef4e7bf
9 changed files with 0 additions and 576 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
docs/Crypto.txt
View File

@ -1,58 +0,0 @@
Draft proposal for how crypto will be implemented.
Encryption library used: http://nacl.cr.yp.to/
When running the program for the first time the crypto_box_keypair() function is used to
generate the users public-private key pair. (32 bytes each)
The generated public key is set as the client_id of the peer.
Adding a friend:
Alice adds bob to her friends list by adding his 32 byte public key (client_id) to her friends list.
2 cases:
case 1: Alice adds Bobs public key and bob waits for Alice to attempt to connect to him.
case 2: Bob and Alice add their respective public keys to their friends list at the same time.
case 1:
Alice connects to Bob and sends a data packet (friends request) like so:
[char with a value of 01][Alice's Public key (client_id) (32 bytes)][Random nonce (24 bytes)][Encrypted message]
Where the encrypted message is encrypted with crypto_box() (using Bobs public key, Alice's private key
and the nonce (randomly generated 24 bytes)) and is a message from Alice in which she tells Bob who she is.
Ex: hello bob it's me alice -_- add me pl0x.
Bob receives the request and decrypts the message using the function crypto_box_open()
If the message decrypts successfully:
If Alice is already in Bobs friends list: case 2
If Alice is not in Bob's friends list: Bob is prompt to add Alice and is shown the message from her.
If Bobs accepts Alice's friends request he adds her public key to his friends list.
case 2:
Bob and Alice both have the others public key in their friends list, they are ready for the next step: Connecting to an already added friend
In the next step.
only crypto_box() is used for encryption and only crypto_box_open() for decryption (just like in the last step.)
Connecting to an already added friend:
Alice and Bob are friends.
As soon as they connect they each generate a new keypair which will only be used for the current connection (The session keys).
They then send themselves the following packet (the crypto handshake) (encrypted part encrypted with the public nonce in the packet
the public key of the receiver and private key of the sender)
[char with a value of 02][Senders Public key (client_id) (32 bytes)][Random nonce (24 bytes)][Encrypted message containing: [random 24 bytes base nonce][session public key of the peer (32 bytes)]]
If the packet is decrypted successfully:
Each start using the secret nonce, the public key provided by the other and their own session private key to encrypt data packets (adding to it + 1 for each packet.)
Each node sends themselves an empty data packet (data packet with 4 encrypted zero bytes)
Data packet:
[char with a value of 03][Encrypted data]
Each data packet received it is decrypted using the secret nonce sent to the other(with +1 added for the first packet +2 for the second, etc...)
along with the private session key of the receiver.
Every data packet sent is encrypted using the secret nonce we received (with +1 added for the first packet +2 for the second, etc...),
the session public key of the receiver and the session private key of the sender.
The encrypted connection is only deemed successful when the empty data packet is received and decrypted successfully.

86
docs/DHT.txt
View File

@ -1,86 +0,0 @@
DHT protocol:
Follows pretty much the principle of the torrent DHT: http://www.bittorrent.org/beps/bep_0005.html (READ IT)
But:
Vastly simplified packet format and encryption.
Boostrapping:
The first time you install the client we bootstrap it with a node on our servers.(bandwidth should not be a problem as the client only needs to be sent one reply.)
Basics (All the numbers here are just guesses and are probably not optimal values):
client list: A list of node ids closest (mathematically see bittorrent doc) to ours matched with ip addresses + port number corresponding to that id and a timestamp containing the time or time since the client was successfully pinged.
"friends" list: A list containing the node_ids of all our "friends" or clients we want to connect to.
Also contains the ip addresses + port + node_ids + timestamp(of last ping like in the client list) of the 8 clients closest (mathematically see bittorrent doc) to each "friend"
Two pinged lists:
-One for storing a list of ips along with their ping_ids and a timestamp for the ping requests
-One for storing a list of ips along with their ping_ids and a timestamp for the get nodes requests
Entries in the pinged lists expire after 5 seconds.
If one of the lists becomes full, the expire rate reduces itself one second or the new ping takes the place of the oldest one.
Entries in client list and "friends" list expire after 300 seconds without ping response.
Each client stores a maximum of 32 entries in its client list.
Each client in the client list and "friends" list is pinged every 60 seconds.
Each client in the client list and "friends" list has a timestamp which denote the last time it was successfully pinged.
If the corresponding clients timestamp is more than 130 seconds old it is considered bad.
Send a get nodes request every 20 seconds to a random good node for each "friend" in our "friends" list.
Send a get nodes request every 20 seconds to a random good node in the client list.
When a client receives any request from another:
-Respond to the request
-Ping request is replied to with with a ping response containing the same ping_id
-Get nodes request is replied with a send nodes reply containing the same ping_id and the good nodes from the client list and/or the "friends" list that are closest to the requested_node_id
-If the requesting client is not in the client list:
-If there are no bad clients in the list and the list is full:
-If the id of the other client is closer (mathematically see bittorrent doc) than at least one of the clients in the list or our "friends" list:
-Send a ping request to the client.
-if not forget about the client.
-If there are bad clients and/or the list isn't full:
-Send a ping request to the client
When a client receives a response:
-Ping response
-If the node was previously pinged with a matching ping_id (check in the corresponding pinged list.)
-If the node is in the client list the matching client's timestamp is set to current time.
-If the node is in the "friends" list the matching client's timestamp is set to current time for every occurrence.
-If the node is not in the client list:
-If the list isn't full, add it to the list.
-If the list is full, the furthest away (mathematically see bittorrent doc) bad client is replaced by the new one.
-If the list is filled with good nodes replace the furthest client with it only if it is closer than the replaced node.
-for each friend in the "friends" list:
-If that friend's client list isn't full, add that client to it
-If that friend's client list contains bad clients, replace the furthest one with that client.
-If that friend's client list contains only good clients
-If the client is closer to the friend than one of the other clients, it replaces the farthest one
-If not, nothing happens.
-Send nodes
-If the ping_id matches what we sent previously (check in the corresponding pinged list.):
-Each node in the response is pinged.
Protocol:
Node format: [char array (node_id), length=32 bytes][ip (in network byte order), length=4 bytes][port (in network byte order), length=2 bytes][Padding , length=2 bytes]
Valid queries and Responses:
Ping(Request and response): [byte with value: 00 for request, 01 for response][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender: [random 8 byte (ping_id)]]
ping_id = a random integer, the response must contain the exact same number as the request
Get nodes (Request):
Packet contents: [byte with value: 02][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender:[random 8 byte (ping_id)][char array: requested_node_id (node_id of which we want the ip), length=32 bytes]]
Valid replies: a send_nodes packet
Send_nodes (response): [byte with value: 03][char array (client node_id), length=32 bytes][random 24 byte nonce][Encrypted with the nonce and private key of the sender:[random 8 byte (ping_id)][Nodes in node format, length=40 * (number of nodes (maximum of 8 nodes)) bytes]]

70
docs/DHT_hardening.txt
View File

@ -1,70 +0,0 @@
List of possible attacks on the current DHT:
create thousands of fake nodes cloning one client_id and flood our DHT with
them.
create thousands of "real" nodes that do nothing but shit up our DHT with fake
crap.
...
Possible solutions:
Each client_id is the public key of the peer so it would be trivial to encrypt
the DHT requests with crypto_box(). This would completely defeat the first
attack.
Make each peer send the information of at least one of his online friends in
every send nodes response. (Might be bad as any node can now know who our
friends are)
Limit the maximum number of peers with identical ips that we keep connected to
in our DHT. (Not a real solution)
Require each node to solve some kind of captcha in order to connect to the
network. (Bad idea.)
Require nodes to crack hashes or solve other computationally intensive
problems in order to be accepted in the network. (Kind of like bitcoin)(This is
probably a bad idea as our application needs to work on phones which are low
power devices)
Make each node test other nodes to see if they respond correctly before sending
them as part of their send nodes response.
...
=====
<slvr> DHT_hardening.txt > create thousands of "real" nodes that do nothing but
shit up our DHT with fake crap.
<slvr> This can be trivially solved by only storing verifiable data in the DHT.
<slvr> there is one attack you have not considered, which is based on the Sybil
attack
<slvr> I am assuming the DHT does say... a hash of a key in order to determine
which node to store data in, similar to Kad?
<slvr> If there happens to be a malicious node at that DHT address, they might
actively deny storing that data.
<slvr> This can be reduced by storing data at multiple places in the DHT
(equidistant points in DHT address space)
<slvr> Since DHT addresses are public keys, it is computationally infeasible for
an attacker to actively deny all storage locations.
<slvr> Recommended reading: S/Kademlia: A Practicable Approach Towards Secure
Key-Based Routing -- http://doc.tm.uka.de/2007/SKademlia_2007.pdf
<biribiri> Type: application/pdf; Size: 202KiB; Updated: 2033d 19h 32m 5s ago
(Tue, 18 Dec 2007 13:28:18 GMT);
<slvr> Tempering Kademlia with a Robust Identity Based System --
http://www.di.unito.it/~ruffo/concorso/Papers/p2p08.pdf
<biribiri> Type: application/pdf; Size: 145KiB; Updated: 1291d 23h 30m 12s ago
(Tue, 29 Dec 2009 09:30:28 GMT);
<slvr> Also of interest: "An Analysis of BitTorrent's Two Kademlia-Based DHTs"
--
http://www.tribler.org/trac/raw-attachment/wiki/AutoUpgradeToLastestVersion/
Measurement_of_Bittorrent_DHT_performance_and_deployed_clients.pdf
<biribiri> Type: application/pdf; charset=iso-8859-15; Size: 1.271MiB; Updated:
1669d 20h 25m 15s ago (Tue, 16 Dec 2008 12:44:08 GMT);

83
docs/FAQ.txt
View File

@ -1,83 +0,0 @@
TOX FAQ
=======
Contents
========
(1). What is Tox?
(2). Where can I get Tox?
(3). Tox
(3.1) Which encryption algorithms does Tox employ?
(3.2) Does Tox have plugin support?
(3.3) I want to contribute to the Tox project.
(3.3.1) I want to be a developer.
(3.3.2) I want to contribute in UI design/sound.
(3.3.3) Are there any other ways I can contribute?
(4). Source
(4.1) Where do I get the Tox source code?
(4.2) How do I compile Tox?
(5). Community
(5.1) Where can I find the latest Tox thread?
(5.2) Are there any other Tox threads/forums?
===============================================================================
(1). What is Tox?
=================
Tox is a free (as in freedom) peer to peer messaging application that aims to
replace skype.
(2). Where can I get Tox?
========================
It's not done yet.
(3). Tox
=======
(3.1). Which encryption algorithms does Tox employ?
--------------------------------------------------
Tox uses the encryption algorithms present in the NaCl crypto library.
(3.2). Does Tox have plugin support?
-----------------------------------
Maybe.
(3.3). I want to contribute to the Tox project.
==============================================
(3.3.1). I want to be a developer.
---------------------------------
Join the IRC.
(3.3.2). I want to contribute in UI design/sound.
------------------------------------------------
Join the IRC.
(3.3.3). Are there any other ways I can contribute?
--------------------------------------------------
Testing the application, reporting bugs and requesting features. Don't be
scared to criticize something if you think it is done wrong.
(4). Source
===========
(4.1). Where do I get the Tox source code?
-----------------------------------------
The core library: https://github.com/irungentoo/ProjectTox-Core
Some front ends:
(None are in a usable state yet.)
(4.2). How do I compile Tox?
---------------------------
You need to build and install libsodium.
Then just cd in the repo and:
mkdir build
cd build
cmake ..
make
(5). Community
==============
(5.1). Where can I find the latest Tox thread?
---------------------------------------------
(5.2). Are there any other Tox threads/forums?
---------------------------------------------

27
docs/IDEAS.txt
View File

@ -1,27 +0,0 @@
A couple of ideas posted in the threads.
#############
Prescence (online/offline)
- A user's id being present with a valid signature in the DHT implies they have been online recently. A ping to the user would confirm this, as well as notifying them that you are online. If both friends in a pair actively search for the other then disruption due to lag in inserting records will be mimimised
Username (nick) changes
- When a user wants to change their nick they are free to do so, the nick is not cryptographically significant. The nick change could be shared in-band with the chat (ie, each message in the format "username - message") or out of band, perhaps in the dht values or in the ping messgaes.
File send
- I see a file being sent as ultimately the same as an audio stream or a video stream being sent, they could use the same protocol parts. The client would handle what to do with the data whether it is a stream of media data or a stream of b64'd file. both would require explicit verification from the other participant. This would also allow sharing of streamed files, eg incomplete downloads.
##############
- ability to log in with username and password
possible implementation: store keys in a server and use user/pass to retrieve keys from server transparently
>too centralized
- looking up people and adding as contacts, etc by name or username
possible implementation: store pubkey-hash/info correspondences in a public directory, integrate access into client
- portable contact list/profile/other account data
just store it along with keys in aforementioned server; cost of storage could rise... potential problem
- POSSIBLY interfacing with regular phones. probably not possible, but plebs might complain (THIS IS EXPENSIVE (hence 'probably not possible')) <3
IMPORTANT: release two major sanctioned UIs, one for autists, one with inbuilt support for the previous list so that plebs can't get confused with setting it up and autists don't complain about it getting in their way. de geso
> I would suggest a "Advanced options" where the autists can rejoice with all kinds of options (and it doesn't frighten the normalfags, since it's not shown by default). Also, 2 UIs would be chaos to maintain.
##############

102
docs/Lossless_UDP.txt
View File

@ -1,102 +0,0 @@
Lossless UDP:
A network protocol over UDP to transfer data without any loss.
Problems with conventional UDP:
-Packet loss.
-Packet mixup.
-No flow control.
Draft of Proposed protocol:
Connexion example:
Alice wants to connect to Bob (Connection handshake):
Alice generates a random 4 byte number.
Alice puts it in the handshake packet (handshake_id1).
Alice starts sending handshake packets to Bob (send 10 packets over 5 seconds if no response connection fails.)
Bob receives the packet.
Bob copies the handshake packet he got from alice but concatenates a random 4 byte number to it (handshake_id2)
Alice receives the packet, checks if handshake_id1 matches the one she sent.
If it does she starts sending SYNC packets with sent_packetnum = handshake_id2 and recv_packetnum = handshake_id1.
Bob receives the packet,
if (handshake_id2) matches Bob adds Alice to his connections list and starts sending SYNC packets to alice.
Alice receives the SYNC packets and starts sending them too.
Connection is now established.
Alice wants to send packets to Bob:
Alice sends packets to bob.
If bob receives the packets, his next SYNC packets reflect it.
Alice receives the SYNC packets which confirms that bob received them
Alice clears the packets from her buffer.
If bob does not receive any packets, but receives a SYNC packet with a larger sent_packetnum than his recv_packetnum:
bobs SYNC packets become request packets.
The packet id of the missing packet(s) are added to the end of the SYNC packet.
Alice resends the packets whose numbers appear at the end of the request packet.
If bob receives some packets but some are missing
bobs SYNC packets become request packets.
The packet id of the missing packet(s) are added to the end of the SYNC packet.
Alice resends the packets whose numbers appear at the end of the request packet.
Alice and bob disconnect suddenly:
Alice stops receiving SYNC packets from bob.
Bob stops receiving SYNC packets from Alice.
Connexion times out if no data is received for 5 seconds.
Packet handling:
The client receives a SYNC packet:
He checks if the packet is valid(sent_packetnum and counter make sense)
If the packet is valid:
If the client was currently trying to establish a connection.
If the handshake was done:
Start sending SYNC packets.
Add +1 to the counter of the SYNC packets we send.
Check to see if any packets were dropped when sending them to us (compare sent_packetnum with the number of packets we received.)
If some were, The next SYNC packets we send will be request packets requesting the missing packets.
Check if the other client received what we sent him (compare recv_packetnum with the number of packets we sent) and clear them from the buffer.
If the packet is a request packet.
Send the packets he requested.
The client receives a Connection handshake packet(not initiating connection):
If handshake_id2 is zero:
use the handshake_id1 in the packet as the handshake_id2 of the response.
use our random handshake_id1 as handshake_id1.
send the response packet.
The client receives a Connection handshake packet(initiating connection):
check if handshake_id1 matches what we sent.
If it does start sending SYNC packets with sent_packetnum as handshake_id2
The client receives a data packet:
Put the packet in the proper place in the receive buffer using packet_num as the reference.
If all the previous packets have been received correctly set recv_packetnum equal to packet_num
Keep track of the percent of packets dropped, if it is too high, lower the send rate. If it is low, increase it.
Have a send packets buffer that we can write to and a received packets buffer that we can read from.
Protocol:
Connection handshake packets:
[byte with value: 16 (10 in hex)][4 byte (handshake_id1)][4 bytes (handshake_id2)]
SYNC packets:
[byte with value: 17 (11 in hex)][byte (counter)][4 byte (recv_packetnum)][4 byte (sent_packetnum)][sequence of requested packet ids[4 bytes each](not present in keep alive)]
data packets:
[byte with value: 18 (12 in hex)][4 byte (packet_num)][data (arbitrary size)]

33
docs/Messenger_Protocol.txt
View File

@ -1,33 +0,0 @@
Protocol for messages, data, etc..
Streaming audio/video will not use this protocol as they can absorb some data loss.
The protocol itself will run on top of the encryption which means it should be
impossible for someone to know what type of data is being transmitted.(Well they
could just analyze how much data is being transmitted for a pretty good guess)
Because it runs on the encryption which itself runs on our Lossless UDP protocol
it can be guaranteed that no data will be lost.
Basic packet format:
[char data_id][data]
data_id represents the type of data.
All strings must be UTF-8.
EX: data_id 64 designates a chat message. so the packet would look like: @Hello WorldNULL
Where @ is the ASCII character for 64, "Hello World" is the message and NULL is the null string terminator.
Proposed data_ids and what they mean (in decimal)
ids 0 to 16 are reserved.
48 Username (Send this packet as soon as you connect to a friend or to each friend everytime you change names.
Username is maximum 128 bytes long with the null terminator included)
49 Status change
64 Chat message
6? File transmission.

66
docs/Routing.txt
View File

@ -1,66 +0,0 @@
Routing Protocol.
The routing protocol will only be used when two clients have difficulty
connecting to each other. This is usually because of NAT issues. For example,
two people who are behind symmetric NATs are not capable of connecting to each
other directly.
The routing protocol severely limits the speed at which two clients can
communicate. This is so that the user of the software does not feel the need to
turn it off because it is taking too much bandwidth and to prevent peers from
using it without good reason.
The routing protocol does not provide any anonymity, only convenience.
#############
Draft of protocol:
Alice wants to connect to Bob.
Alice queries the DHT and manages to obtain the ip_port of Bob from Carol and
Dan both of who are closest mathematically to Bob in the DHT.
Unfortunately Alice is enable to connect to the ip_port for Bob provided by
Carol and Dan.
Alice assumes then that Bob is behind a bad NAT.
Alice therefore randomly picks between Carol and Dan. She picks Carol.
Alice connects to Carol using the Lossless UDP protocol.
She then sends a routing request over the connection:
[char with a value of 16][Public key of who to route the packets to(client_id)
(32 bytes)]
Carol checks if she is connected via the DHT to the person (Bob) who the public
key appears in the routing request.
If she is not she kills the connection.
If she is, she waits for the next data packet to arrive from Alice.
As soon as she receives it she connects to the person (Bob) and sends it to him.
If nothing is received from Bob within a timeout, the connection is killed.
If something is received from Bob, it is sent to Alice and the connection is
confirmed and continues until either Bob or Alice disconnect.
#############
Some notes:
If both Alice and Bob are friends they will create two different connections
when each try to connect to each other witch is good because it means data can
be sent/received on both which lower the chances of the connection being
severed because the node shut itself down or data being lost because of a bad
node. It however doubles the amount of data we send/receive.
If both peers manage to connect to each other, the routing connection is
killed.
All data transmitted trough this protocol must be encrypted in a way that it is
only decryptable by the receiver and only the receiver.

51
docs/TODO.txt
View File

@ -1,51 +0,0 @@
Things to do now:
-Network protocol (Done)
-Figure out the best way to do "lossless" UDP. (Done)
-Start work on the im protocol.(simple im part pretty much done)
-Start coding the gui (In Progress (Using Qt5))
-Get a basic im client working using the now completed DHT implementation to find the ips of your friends.
-Find some good encryption libraries.(Done)
we will use: http://nacl.cr.yp.to/
-Add NaCl to our build system.
-Make NaCl work on windows (DONE)
https://github.com/jedisct1/libsodium
-Crypto (Done (needs testing))
-Harden the DHT (Research in progress)
-Find and fix bugs in the code.
Things to do later:
-Figure out the whole sound and video transmission. (encrypted and fast)
-File transfer (encrypted and fast)
Less important.
-Symmetric NATs
No UDP hole punching on them so we need to do something else
(only if both the clients which try to connect to themselves are behind one)
-Decentralized IRC like channels (chatrooms).
-Offline messaging protocol (text only)
Use your friends.(or maybe the people closest (mathematically by comparing client_id's) to you or the friend you want to send the message to).
The message will not be very big. Lets say we limit the maximum number of bytes for one to 1024, it means if every client stores 1024 offline messages it only takes 1 MB of ram.
-IPv6
Currently the core only supports ipv4