Commit Graph

638 Commits

Author SHA1 Message Date
Robin Linden
1937ee39c3
Release 0.2.16 2022-02-19 00:51:01 +01:00
iphydf
c5e3bca6de
chore: Simplify and speed up nacl build using toxchat/nacl. 2022-02-18 22:13:42 +00:00
iphydf
4389c71619
test: Add a profiling script and Dockerfile.
This works on my kernel. If your kernel is older/newer, you may need a
different alpine base image that matches your kernel more closely.
2022-02-18 21:38:55 +00:00
iphydf
84a03bc2ed
cleanup: Enable tokstyle's -Wlarge-struct-params. 2022-02-18 10:49:46 +00:00
iphydf
cc045d6343
cleanup: Avoid implicit boolean and floating point conversions in decls.
Also avoid implicit enum-to-int.
2022-02-18 03:26:12 +00:00
jfreegman
0f3534656f
Improve how we share TCP relays with friends
Previously we would try to send three random TCP relays that we're
connected to to each friend once every 5 minutes. The problem with
this method is that it could take an extraordinarily long time
to share every relay; some relays might be consistently skipped
while others might be sent repeatedly. Moreover, there's no
guarantee that the nodes you try to send are actually online.
This leads to a prety unreliable and flaky way of sharing.

Now we reduce the timer to two minutes, and cycle through the list
trying 3 nodes each share attempt. This guarantees that every online
node in our list gets shared with every friend after a fixed amount of
time (which depends on how many nodes are in the list)
2022-02-17 16:16:29 -05:00
iphydf
5e724d8909
cleanup: Avoid implicit pointer-to-bool conversion in if in toxav. 2022-02-17 20:33:24 +00:00
iphydf
933c6b7517
refactor: Compare pointers in if conditions to nullptr.
Don't use implicit ptr-to-bool conversion.
2022-02-17 19:55:29 +00:00
iphydf
43fc2374bc
refactor: Add a bit more logging; change WARNING to ERROR.
ERROR is caught in tests, so we'll use it more for things that ought not
to happen.
2022-02-16 23:05:10 +00:00
iphydf
1d0f6e51e2
cleanup: Add some more error path logging to TCP server code. 2022-02-15 12:08:22 +00:00
iphydf
c9d3e078e2
chore: Add BUILD file for websockify. 2022-02-14 13:56:18 +00:00
Maxim Biro
dcc65c385f
Update Windows Docker build deps 2022-02-13 22:54:08 -05:00
iphydf
a3cd1102f7
perf: Reduce minimal encoding size of packed events.
We're using a union-like encoding now with an enum telling which union
member to set.
2022-02-12 17:44:44 +00:00
iphydf
33588c1aad
cleanup: Don't use VLAs for huge allocations.
One of these was creating a single 262144 byte stack frame. We now have
a way to check and limit the allocation size of a VLA. The `Cmp_Data`
ones were also fairly large. Now, no allocation is larger than 2KiB
(though rtp.c allocates close to that much).
2022-02-11 03:05:30 +00:00
jfreegman
73ba0504b0
Add autotest for dht getnodes API
The test creates a small DHT and makes sure that each
peer is able to crawl the full network
2022-02-10 10:33:34 -05:00
jfreegman
652ae27eeb
Add DHT queries to private API
This commit adds functionality for clients to interact with
the DHT, sending getnodes requests to their peers and receiving
nodes in getnodes responses.
2022-02-10 10:33:31 -05:00
Maxim Biro
2965403e2c
Add bash-completion for tox-bootstrapd 2022-02-10 02:40:42 -05:00
iphydf
10f86f6c00
cleanup: Add more nonnull and nullable annotations. 2022-02-09 20:03:34 +00:00
iphydf
5a3a0b6453
cleanup: Add nonnull annotations to function definitions.
These help static analysis and ubsan. We should eventually have all
functions annotated like this with a cimple check to make sure every
pointer has an explicit nullability annotation. The `nullable`
annotation does nothing in GCC, but will be used by cimple to validate
that every parameter has defined nullability.
2022-02-08 17:57:47 +00:00
iphydf
6f36b67a13
chore: Add an undefined behaviour/integer sanitizer build. 2022-02-08 09:18:18 +00:00
iphydf
0a2190f28c
feat: Add binary packing functions for tox events. 2022-02-07 02:43:24 +00:00
iphydf
9218566599
cleanup: Make LAN discovery thread-safe without data races.
It was kind of thread-safe, maybe, but there was a data race that makes
tsan unhappy. We now do interface detection once per Tox instance
instead of once per process.
2022-02-06 19:56:34 +00:00
iphydf
de4af4c270
feat: Add async event handling (callbacks) code.
Instead of synchronously handling events as they happen in
`tox_iterate`, this first collects all events in a structure and then
lets the client process them. This allows clients to process events in
parallel, since the data structure returned is mostly immutable.

This also makes toxcore compatible with languages that don't (easily)
support callbacks from C into the non-C language.

If we remove the callbacks, this allows us to add fields to the events
without breaking the API.
2022-02-06 17:28:28 +00:00
iphydf
cde796f1f8
cleanup: Comply with new cimple callback rules.
* Function arguments must use `foo_cb *p` and can't just use `foo_cb p`
* You can no longer cast function pointers (if it's incompatible, you
  must wrap the callback). I'm avoiding this with tokstyle exclusions.
2022-02-06 16:45:41 +00:00
iphydf
4d4120214a
chore: Add an easy way to run autotools and circleci builds locally. 2022-02-06 15:48:02 +00:00
Robin Linden
d93e795cfe
Release 0.2.15 2022-02-06 04:40:03 +01:00
iphydf
79eb17aeac
fix: Reduce logging verbosity even more. 2022-02-06 03:33:34 +00:00
iphydf
fedcdea95a
fix: Reduce logging verbosity in TCP server. 2022-02-06 03:06:27 +00:00
Robin Linden
f41caa2a1e
Release 0.2.14 2022-02-05 22:24:58 +01:00
iphydf
d39f803c7e
docs: Make crypto_core.h appear on doxygen. 2022-02-05 19:34:13 +00:00
jfreegman
96814c4645
Use proper method for generating random numbers in a range
Using modulus creates a bias in its output. The libsodium
randombytes_uniform() function guarantees a uniform distribution
of possible outputs
2022-02-05 11:14:45 -05:00
iphydf
2ed9b1927f
doc: Add doxygen configuration and netlify publishing. 2022-02-05 15:59:45 +00:00
iphydf
1859d0f44a
cleanup: Ensure we limit the system headers included in .h files.
Most system headers contain functions (e.g. `memcpy` in `string.h`)
which aren't needed in our own header files. For the most part, our own
headers should only include types needed to declare our own types and
functions. We now enforce this so we think twice about which headers we
really need in the .h files.
2022-02-04 20:54:37 +00:00
Maxim Biro
cda6c9b6e8
Fix potential freeing of an immutable static buffer
strerror_r() has two versions: GNU-specific and XSI-compliant. The XSI
version always stores the string in the provided buffer, but the GNU
version might store it in the provided buffer or it might use some
immutable static buffer instead. Since we always free the error string,
we might end up freeing the immutable static buffer.
2022-02-04 08:09:14 -05:00
iphydf
28dc8c1ded
cleanup: Don't pass the whole DHT object to lan discovery.
It only needs net and dht public key.
2022-02-03 20:50:36 +00:00
iphydf
014cc42249
chore: Make run-infer script use docker.
So people can run it locally more easily.
2022-02-03 20:11:16 +00:00
jfreegman
1d3778fc53
Move duplicate code for converting IPv4-in-6 to IPv4 to a function 2022-02-03 12:32:30 -05:00
jfreegman
5619b5e056
Make more functions take const pointers to IP_Port
We additionally now make local copies of the IP_Port param instead
of modifying the passed argument
2022-02-03 11:17:29 -05:00
jfreegman
5880971af8
Make functions take const pointer to IP_Port wherever possible
An IP_Port is a fairly large data structure, and copying them down
the call stack creates a lot of unnecessary overhead.
2022-02-03 10:54:48 -05:00
jfreegman
3a72ed5fae
Properly copy Node_format using serialized format
Also fix a bad comment
2022-02-02 22:00:06 -05:00
iphydf
71965a0f3b
feat: Add WASM build for toxcore and websocket bootstrap node.
The websocket bootstrap node will be running on Heroku.
2022-02-02 16:05:25 +00:00
iphydf
727982d2f9
fix: Fix bootstrap on emscripten/wasm.
Also added a whole bunch of logging that I needed while debugging the
issue. The solution in the end is that bootstrap needs to resolve IPs,
and getaddrinfo fails in the browser. Most of the time we bootstrap
against IPs anyway, so trying to parse as IP address first will shortcut
that.
2022-02-01 21:00:09 +00:00
jfreegman
52c31efb97
Small refactor of DHT getnodes function 2022-01-29 16:52:25 -05:00
jfreegman
3f4ea3d5fd
Remove brackets from ip_ntoa ipv6 formatting
The brackets serve no purpose and make us do extra string
parsing when using the output for other things

Also removed a useless call to ip_ntoa in LAN_discovery.c
2022-01-29 15:29:14 -05:00
jfreegman
8c3bd7fc35
Fix some unintentional integer down-casts
None of these currently cause any bugs from what I
can see, but they're still incorrect.
2022-01-28 09:57:17 -05:00
Robin Linden
2cfa872818
Add a Bazel Buildifier CI job 2022-01-28 00:55:07 +01:00
Maxim Biro
b2ea0e5c76
Add programs for creating savedata & bootstrap keys 2022-01-27 16:31:07 -05:00
Tha14
5a88159b8f
Pass "b" for fopen to fix bugs on windows 2022-01-24 16:00:01 +02:00
iphydf
7f94e411a9
chore: Add cpplint to the CI. 2022-01-17 23:42:38 +00:00
jfreegman
fc18810ba8
Fix buffer overwrite in bootstrap config 2022-01-17 17:56:47 -05:00
iphydf
fdadcb0a90
chore: Add a make_single_file script, used for CI.
This will also be used for static analysers that can't deal with multiple
source files.
2022-01-17 21:12:25 +00:00
iphydf
9dfad80017
cleanup: Remove our only use of flexible array members in toxcore.
We still have them in toxav. That will need to be cleaned up later.
Flexible array members have very limited usefulness. In this particular
case, it's almost entirely useless. It confuses static analysers and is
yet one more C feature we need to understand and support. It is also the
only reason we need special support in tokstyle for calloc with a `+`
operator in the member size.
2022-01-17 18:37:16 +00:00
iphydf
5fbcbb6c83
cleanup: Remove uses of strcpy and sprintf.
Use of `strcpy` in these particular cases was safe, but it's hard to
tell and also useless. `strcpy` would effectively need to do another
`strlen` which we already did.

Also removed sprintf, which was also safe in this case but it's easier to
be "obviously safe", especially for static analysers.
2022-01-17 18:25:40 +00:00
iphydf
4ce02c0af9
refactor: Deduplicate a bunch of code in TCP client/server.
Also generally a bit of cleanup and better layering.
2022-01-16 09:41:08 +00:00
iphydf
044ae636a3
chore: Mark unsafe code as testonly.
We want to ensure that nobody links against testing code in production.
2022-01-16 09:10:53 +00:00
iphydf
422c8d1125
cleanup: Make parameters pointers-to-const where possible. 2022-01-16 01:28:25 +00:00
iphydf
09bb9b8a23
cleanup: Make Networking_Core pointer-to-const where possible. 2022-01-16 01:09:44 +00:00
iphydf
c081a50201
cleanup: Use pointer cast instead of memcpy in qsort callback.
No need to make copies of the data. There is no concurrency in qsort.
2022-01-16 00:12:44 +00:00
iphydf
0d27eb2f90
cleanup: Deduplicate a somewhat complex loop in DHT.c. 2022-01-15 21:25:14 +00:00
iphydf
2671095f4a
cleanup: Merge crypto_core and crypto_core_mem.
The remaining memory functions are small and don't need their own file.
2022-01-15 01:34:14 +00:00
iphydf
6823fcbae1
cleanup: Stop using strerror directly.
We have a more portable wrapper that is now also thread-safe. Also
stopped using sprintf in the one place we used it. This doesn't really
help much, but it allows us to forbid sprintf globally.
2022-01-15 01:25:56 +00:00
iphydf
8d19757f8e
chore: Add mypy Python type check.
Also fix the types in all Python files.
2022-01-14 19:55:37 +00:00
iphydf
d23222c92f
chore: Run static analysers in multiple variants.
Currently: 1) libsodium and 2) nacl.

Note that the "nacl" variant is actually libsodium. We just want to make
sure the static analysers see the `VANILLA_NACL` code paths.
2022-01-14 10:45:11 +00:00
iphydf
4a2cb37e4b
fix: Fix some uninitialised memory errors found by valgrind and msan.
Also added a valgrind build to run it on every pull request. I've had to
disable a few tests because valgrind makes those run infinitely slowly,
consistently timing them out.
2022-01-13 20:15:21 +00:00
jfreegman
360acd0f42
Replace all instances of atoi with strtol
atoi doesn't check if the conversion from string to int succeeded
which doesn't allow us to do proper error handling.

We also now make sure that the port argument is a valid port
in addition to being properly converted
2022-01-12 14:08:45 -05:00
iphydf
f5f22a5c76
chore: Enable layering check in all c-toxcore build files. 2022-01-12 18:33:13 +00:00
iphydf
cfb0aa8f25
cleanup: Remove extra parens around function arguments. 2022-01-12 07:02:29 +00:00
iphydf
c1b32bd7a1
chore: Enable compiler layering check.
This ensures that we're explicit about what we depend on and avoids
transitive dependencies.
2022-01-11 15:26:32 +00:00
iphydf
eebc516d55
chore: Disable the OpenMP cracker in bazel for now.
I broke it. I'll re-enable it if/when I fix omp support in the bazel
build.
2022-01-11 13:29:53 +00:00
sudden6
faf39ef94f
update tox-bootstrapd.sha256
__LINE__ macros were changed due to inserting #ifdef
2022-01-10 21:05:51 +01:00
sudden6
e04b890817
add scripts to run the fuzzing process
This adds scripts and Dockerfiles to run the fuzzing process standalone
or with OSS-Fuzz/ClusterFuzzLite integrations.
2022-01-10 21:05:51 +01:00
iphydf
f46f51e554
cleanup: Don't use memcpy where assignment can be used.
See https://github.com/TokTok/hs-tokstyle/pull/104.
2022-01-10 10:58:42 +00:00
Robin Linden
23df660fd2
Remove always-true check in receivepacket error handling 2022-01-10 02:27:13 +01:00
iphydf
5ab301ecab
chore: Remove all references to Travis CI.
Travis gave up on FOSS, so no more free advertisement for Travis.
2022-01-10 01:17:46 +00:00
iphydf
73484d8995
cleanup: Use calloc instead of malloc for struct allocations.
`malloc` is used only for byte/int arrays. Also, we no longer allow plain
`void *` allocations. Every allocation should have a real value type.
2022-01-09 22:42:50 +00:00
iphydf
83d5d0d280
cleanup: Remove useless parentheses. 2022-01-06 20:36:03 +00:00
iphydf
e848853968
chore: Remove old travis docker scripts.
These aren't useful anymore. We don't have travis anymore, so there's no
need to simulate the travis CI environment.
2022-01-02 19:41:46 +00:00
jfreegman
7318127dca
Remove dead code
Neither of these clauses can ever be reached.

Also attempt to fix travis Windows build
2022-01-01 19:44:11 -05:00
iphydf
8cd4827a14
cleanup: Reduce the scope of for-loop iterator variables.
Found by tokstyle after https://github.com/TokTok/hs-tokstyle/pull/95.
2022-01-01 03:26:41 +00:00
iphydf
6f61de54d3
chore: Add "tcc" and "compcert" compiler targets.
It's nice we are able to compile with `tcc`. Let's not break that.
CompCert is also neat, but its interpreter mode doesn't work on tox, so
we only use the compiler.
2021-12-30 23:09:23 +00:00
iphydf
1e11865c4c
chore: Add bootstrap daemon docker image build.
This will also push to dockerhub on merge.
2021-12-23 15:54:24 +00:00
iphydf
ac5df0692c
chore: Add docker image for running circleci/cmake-{asan,tsan}.
Use `make asan` or `make tsan` to run a similar setup to the one we run
on circleci.
2021-12-22 10:14:16 +00:00
iphydf
74bf0fc2c8
chore: Disable -Wmissing-braces.
This flag isn't helpful for the most common case of aggregate
initialisation, namely `{0}`. We don't want to be writing `{{{0}}}` with
some brittle number of braces.
2021-12-21 12:04:04 +00:00
iphydf
4746a8b3af
cleanup: Remove apidsl; remove crypto_memcmp.
* `crypto_memcmp` was replaced by more specific functions. We never want
  to compare arbitrary amounts of data this way. We use these functions
  to compare key material.
* apidsl has been bothering people, so now we un-bother them. You're
  welcome.
* Added the memlock/unlock functions from the New Group Chats branch.
* Remove some system dependencies in crypto_core_mem.c.
* Renamed UPPERCASE_NAMES to Snake_Camel_Case names.
2021-12-19 19:22:38 +00:00
jfreegman
a42b0d62ae
Fix format-source
APIDSL no longer works with these header files
2021-12-17 16:00:59 -05:00
sudden6
b14d9328b4
update build scripts 2021-12-17 20:34:13 +01:00
sudden6
828312c2f9
perf: improve speed of cracker.c
- more inteligent secret key generation
- use OpenMP for multithreading
- leave old cracker, because it's simple
- update old cracker to new coding style
2021-12-17 20:11:18 +01:00
iphydf
2ba2cbd8e9
chore: Fix gen-file.sh: it wasn't globbing properly.
Also fix crypto_core_test.cc to comply with the stack frame limit.
2021-12-14 21:26:52 +00:00
iphydf
cd83c9282c
chore: Add .cc files to the static analysis. 2021-12-14 19:33:18 +00:00
iphydf
5c1796d3c4
chore: Use docker for the autotools ci build.
This makes it easier to run it locally for testing and ensures the
dockerfile continues to work.
2021-12-14 15:51:51 +00:00
Maxim Biro
064ec836fe
Make CodeFactor happy 2021-12-14 08:55:53 -05:00
Maxim Biro
805858716e
Fix file permission issue in toxchat/bootstrap-node
/var/lib/tox-bootstrapd on the host is owned by hosts's tox-bootstrapd
and chowned 700, but the container attempts to access it as its own
tox-bootstrapd user with possibly different uid:gid, which will fail if
host's tox-bootstrapd user has different uid:gid than the tox-bootstrapd
user inside the container.

This change makes the container use host's tox-bootstrapd uid:gid, which
fixes the issue.
2021-12-14 08:55:52 -05:00
Maxim Biro
0538273479
Update base images of toxchat/bootstrap-node 2021-12-14 08:55:48 -05:00
iphydf
30c939e4ab
cleanup: Fix some clang-tidy warnings and make them errors.
The android warnings are disabled now because they suggest using
linux-only extensions of libc. Useful for android indeed, but we're
targeting non-android and non-linux systems as well.
2021-12-10 15:21:27 +00:00
iphydf
fd91bbdd7b
test: Add a simple test for ip_ntoa.
Just to demonstrate that it will never exceed its input buffer.
2021-12-09 20:51:15 +00:00
Maxim Biro
4e8e556a93
Fix previous refactor
Includes should be sorted alphabetically and null-terminating is
unnecessary with snprintf.
2021-12-09 13:23:06 -05:00
jfreegman
edf9d66717
Add logger to onion and onion announce objects
This will be used in the future
2021-12-07 13:01:15 -05:00
iphydf
1ce6aab5da
cleanup: Ensure that error codes are always initialised.
In this case, there was no way it would not be, but a code change down
the stack could cause a variable to become uninitialised. This avoids a
gcc warning and is more locally-correct.
2021-12-07 15:12:42 +00:00
iphydf
101e235131
cleanup: Fix last instance of -Wcast-align and enable the warning.
We used to have lots of these in the code, but now that all the endian
stuff is no longer dependent on host byte order, we can re-enable the
warning flag and catch any future violations.
2021-12-07 14:51:31 +00:00
iphydf
2db6599be5
test: Add some simple tests for list.c.
These are pretty useless, but I'm adding the list_test.cc file for future
better tests to be written in.
2021-12-07 10:32:45 +00:00