Commit Graph

760 Commits

Author SHA1 Message Date
iphydf
41e6ea865e
cleanup: Use tcc docker image for CI.
Instead of repeating the instructions in github actions configs.
2023-11-18 03:44:06 +00:00
iphydf
e726b197b0
refactor: Store time in Mono_Time in milliseconds.
Conversion to seconds happens in `mono_time_get`, and a new function
`mono_time_get_ms` allows code to retrieve monotonic time in
milliseconds.
2023-11-17 05:20:29 +00:00
zoff99
cd34b60f0f
feat: allow for larger incoming NGC packets 2023-11-16 20:47:02 +00:00
iphydf
94cf9d1f36
fix: Fix memory leak in the error path of loading savedata. 2023-11-16 00:41:45 +00:00
jfreegman
fc623a5281
tox_new() should return null when savedata loading fails
Returning a valid tox instance when loading a corrupt
savefile is probably not desired behaviour
2023-11-14 22:15:55 -05:00
Tha14
06d949a701
fix: always respond to version packets with toxcore version 2023-11-13 15:45:17 +00:00
Green Sky
8099d82397
diagnostic: get the number of close dht nodes with announce/store support 2023-11-13 01:24:15 +01:00
iphydf
58fac53429
refactor: Add a bin_unpack_bin_max for max-length arrays.
These are statically allocated (e.g. `uint8_t[1024]`) arrays with
variable length data inside them. Examples are group topics and
nicknames.
2023-11-12 00:57:44 +00:00
iphydf
6be29f01e5
chore: Add more logging to loading conferences from savedata.
Better error messages than "something went wrong in type 20".

Also fix bazel-asan/tsan builds.

https://github.com/tweag/rules_nixpkgs/issues/442 is blocking fuzz
tests under asan.
2023-11-08 15:51:11 +00:00
jfreegman
1195271b7f
Fix inversed return values
This was causing pings to be sent every 1-2 seconds rather than
the expected interval.
2023-11-07 12:54:20 -05:00
iphydf
24b54722ae
fix: Ensure we have allocators available for the error paths. 2023-09-11 21:43:43 +00:00
iphydf
0cef46ee91
cleanup: Fix a few more clang-tidy warnings. 2023-09-07 07:49:47 +00:00
iphydf
0c5b918e9f
cleanup: Fix a few more clang-tidy warnings. 2023-09-07 07:06:34 +00:00
iphydf
4d3c97f49d
cleanup: Enforce stricter identifier naming using clang-tidy. 2023-08-31 13:37:16 +00:00
iphydf
a549807df7
refactor: Add mem module to allow tests to override allocators.
This will allow us to do more interesting things with memory allocation
within toxcore, and allow fuzzers to explore various allocation failure
paths.
2023-08-30 22:55:05 +00:00
iphydf
a1e2458412
docs: Fix doxygen config and remove some redundant comments.
Doxygen doesn't like these. We should probably remove all redundant
comments, but this was the set of comments doxygen complained about.
2023-08-16 07:53:19 +00:00
Green Sky
2b1a6b0d23
add missing ngc constants getter declarations and definitions 2023-08-14 12:47:08 +02:00
iphydf
2e02d56379
chore: Add missing module dependencies.
Also flip some callback asserts, because they can be reached by fuzzing
eventually.

Also update the bootstrapd checksum, since the alpine image changed a
bit.
2023-08-10 21:01:17 +00:00
jfreegman
172f279dc0
Increase max group message length by four bytes
The max message length was reduced by 4 bytes to account for the pseudo message ID, which had unintended effects on clients. It makes more sense to increase the raw packet length by four and keep the max group message length the same as the max message length for friend chats.
2023-03-03 15:53:22 -05:00
jfreegman
b2ca401543
Fix behaviour of group api function
The function that tells us if we're connected to a group now behaves
according to the documentation and returns true if we're attempting
to connect to a group, rather than only returning true if we've
connected with other peers
2023-01-24 17:32:31 -05:00
Green Sky
88ffd1a649
fix: resolve_bootstrap_node() not checking net_getipport() returned count correctly
doc: improve inline docu of return of net_getipport
2023-01-04 00:03:15 +01:00
sudden6
873896058f
chore: update tox-boostrapd hash 2022-12-08 13:16:14 +01:00
sudden6
ad2375ffe9
fix: potential endless loop under extremely high load
If there are more connections waiting to be accepted than
MAX_INCOMING_CONNECTIONS existing accepted connections are overwritten.
2022-12-05 23:40:23 +01:00
jfreegman
7ee7720890
Make some improvements to how often/when we announce a group
Instead of announcing a group whenever our connection status changes,
including when we gain or lose a TCP relay connection, we now only
when our UDP status changes, or if our previously announced
relay has gone offline. We also refresh our announcement
20 minutes regardless of any connection changes.

change should vastly reduce the amount of unnecessary DHT
traffic related to group announcements.
2022-11-25 10:56:47 -05:00
jfreegman
b8aa21cc44
Fix group custom packet size limits
Lossy custom packets cannot be split, therefore they need to be limited
to the maximum safe UDP packet size.
2022-11-24 12:18:14 -05:00
sudden6
8c7d30475f
refactor: re-implement shared key cache in separate file
The exisiting implementation is not clearly documented and used by
multiple modules.
2022-11-22 23:43:05 +01:00
Maxim Biro
e0b00d3e73
Make Restyled/shellharden happy 2022-10-07 12:17:50 -04:00
Maxim Biro
933a2aa452
Disambiguate the provided to cmake path to be a src dir
cmake treats the provided path differently depending on whether it
contains a CMakeCache.txt or not. If it doesn't contain it -- it's
treated as a path to the source tree, if it does -- as a path to the
build tree. We want it to be treated as a source tree path, but if a
user has CMakeCache.txt in that directory, e.g. from a previous in-tree
build the user has done, cmake will treat it as a build tree instead,
which might lead to unexpected results (improperly configured build) or
an error, with the latter being more likely considering we are building
inside a container and the host paths specified in the user-generated
CMakeCache.txt likely don't exist in there.
2022-10-07 11:46:18 -04:00
Green Sky
7dfa935ac8
fix: missing net to host conversion of port in logging in group_chat.c 2022-10-04 03:41:17 +02:00
jfreegman
856e4fed8a
Fix bug causing friend group invites to sometimes fail & improve logging
The group privacy status was incorrectly set to private when a peer
accepted a friend's group invite, which would cause handshake requests
to fail in certain scenarios
2022-10-01 14:40:35 -04:00
Green Sky
8054854b58
fix: missing net to host conversion of port in logging 2022-10-01 20:22:11 +02:00
Tha_14
fb99aea9e3
Update README for bootstrap node docker 2022-09-28 00:34:13 +03:00
jfreegman
0a277b52ea
Merge the remainder of the new groupchats implementation
Commit history: https://github.com/jfreegman/toxcore/tree/ngc_jf
Spec: https://toktok.ltd/spec.html#dht-group-chats
2022-09-22 11:16:08 -04:00
Green Sky
d3819b23b3
refactor: extract each case in handle packet in messenger 2022-09-19 20:44:16 +02:00
sudden6
306693aba3
fix: unlock correct dht_friend 2022-08-07 15:37:10 +02:00
sudden6
9a265804f8
chore: update bootstrapd hash 2022-05-25 21:04:21 +02:00
sudden6
008d9c89ce
test: disable cppcheck rules that trigger false positives 2022-05-25 21:00:15 +02:00
sudden6
80c3a5dbb7
test: remove outdated cppcheck option 2022-05-25 20:43:00 +02:00
iphydf
7205db72e4
cleanup: Remove check_recursion, since tokstyle checks this now.
And tokstyle is better at it, covering not just direct recursion, but
also (most possible instances of) recursion through function pointers.
2022-05-24 21:35:16 +02:00
iphydf
50094b7385
test: Add a protocol dump test to generate initial fuzzer input. 2022-04-19 23:42:47 +00:00
Robin Linden
3a5da3588f
Release 0.2.18 2022-04-18 20:12:07 +02:00
zoff99
6b0d1151ac
fix: remove the assert because buffer can be larger than UINT16_MAX. 2022-04-18 17:14:02 +02:00
zugz (tox)
452a02f729
cleanup: remove unused field last_seen from Onion_Friend 2022-04-17 11:01:37 +00:00
iphydf
19b9cf3812
test: Enable fuzzing for TCP.
If the `recvbuf` network function returns 0 all the time, that means
there is never any data available on the TCP socket. This change makes
it so there is a random amount of data available on the TCP socket.

This invalidates the bootstrap fuzzer corpus.
2022-04-14 13:19:56 +00:00
iphydf
efea12ed6f
Revert "fix: Allow onion paths to be built from more random nodes."
This reverts commit 5073882e0f.
2022-04-11 11:56:43 +00:00
iphydf
365ecd57f2
refactor: Move crypto utilities from util to crypto_core.
This makes more sense as a module for them to live in. Now, util no
longer depends on crypto_core and can thus potentially be used in
crypto_core in the future (functions like min/max may be useful).
2022-04-10 23:27:15 +00:00
iphydf
d78ee9b12e
fix: Fix potential array out-of-bounds in DHT random node retrieval.
It can't happen in almost every reality, except when the RNG is fairly
broken and doesn't add 2 fake DHT friends on startup. Still, this code
should be defensive and never index outside `num_friends` elements.
2022-04-10 23:17:15 +00:00
iphydf
5073882e0f
fix: Allow onion paths to be built from more random nodes.
Right now it only gets built from the first 2 friends in the DHT friend
list: either friend 0 and then 1 or friend 1 and then 0. The randomness
in this code doesn't make sense unless the intention was to select from
all friends, which the code will now do.

Also: use uniform random distribution to select the friends rather than
modulus, which is only uniform for powers of 2.
2022-04-10 22:10:32 +00:00
zugz (tox)
417d28d439
fix: really fix overrun added in e49a477a 2022-04-10 00:00:00 +00:00
jfreegman
7e38ead415
Add missing return on error
Note: This code cannot presently be called
2022-04-08 19:23:12 -04:00
zugz (tox)
697e8d0264
fix leak and overrun added in e49a477a 2022-04-07 00:00:00 +00:00
iphydf
cb34fe12d1
test: Add more functionality to the bootstrap harness.
Ideally this would be able to reach some of the events, so we can write
code to respond to those events, but so far only the friend request
event actually happens.
2022-04-07 20:17:56 +00:00
jfreegman
f68b1412b4
Merge group announce portion of new groupchats implementation 2022-04-06 10:56:09 -04:00
iphydf
2fb25b9328
refactor: Protect array unpacking against invalid lengths.
Each array element is at least 1 byte, so if there are fewer bytes than
array elements, the array size is invalid.
2022-04-05 20:52:47 +00:00
iphydf
c4beda4dd1
refactor: Rename announce functions into their own namespace.
This avoids common names like `on_stored` and `store_data` in global
symbols. Turns out, toxic also has a `store_data`.
2022-04-05 20:30:53 +00:00
iphydf
d539e34f91
chore: Add initial ESP32 docker build.
It won't actually run until we reduce memory consumption by 2 orders of
magnitude, but that'll be the eventual goal.
2022-04-05 20:11:18 +00:00
iphydf
3576df9d79
chore: Add MISRA-2012 check using cppcheck's misra addon.
Disabled a whole bunch of rules from the MISRA-C set. Some of them
should be fixed, but most of the ones we violate have good reasons. This
PR documents those reasons.
2022-04-04 20:29:54 +00:00
iphydf
d0728560bc
fix: Format IP as string again in error log.
In case DEBUG logging is disabled, the formatting done in the
`LOGGER_DEBUG` statement above will not occur, leaving uninitialised
memory here.
2022-04-04 10:37:32 +00:00
iphydf
7a4cc107c0
fix: Fix a stack overflow triggered by small DHT packets.
This isn't in production yet. It's in the new announce store code. The
problem was that a negative plain_len was converted to unsigned, which
made it a very large number.
2022-04-04 09:35:19 +00:00
iphydf
2c06ef6ad4
cleanup: Replace a series of if statements with a switch. 2022-04-04 09:02:48 +00:00
iphydf
941026266e
refactor: Allow overriding mono_time in tox_new.
This makes it so if mono_time is overridden, no monotonic time-related
system call is invoked in tox_new.
2022-04-03 22:48:16 +00:00
Robin Linden
657d185de5
chore: Support producing shared libraries on Windows
The FLAT_OUTPUT_STRUCTURE CMake option was added to allow the tests to
find the .dlls.

The global data symbols were converted to functions to allow
WINDOWS_EXPORT_ALL_SYMBOLS handle them.

See:
https://cmake.org/cmake/help/v3.23/prop_tgt/WINDOWS_EXPORT_ALL_SYMBOLS.html
2022-04-03 23:04:11 +02:00
iphydf
ddda605509
fix: Don't crash if RNG init failed.
`system_random()` can fail and return NULL, which should be handled by
toxencryptsave functions.

Also synced function comments between .h and .c file for toxencryptsave.
2022-04-03 20:46:44 +00:00
zugz (tox)
4430515052
cleanup: remove populate_path_nodes_tcp
It was a no-op.
2022-04-03 00:00:00 +00:00
iphydf
7a3ead591f
cleanup: Use a struct for the ip_ntoa buffer.
Every use of this function needs to allocate the same buffer. None of
the callers uses a differently sized buffer, so we might as well put it
in a struct and have the type checker prove the buffer size is correct.

Also rename `ip_ntoa` to `net_ip_ntoa` to avoid clashes with ESP-IDF
system libraries which define this function as well.
2022-04-03 18:57:04 +00:00
iphydf
e4d1958ffa
refactor: Allow NULL logger; make it no-op in NDEBUG. 2022-04-03 18:29:22 +00:00
iphydf
378febffff
cleanup: Use static_assert instead of assert where possible.
This avoids some "always true condition" warnings and lifts the errors
(if any) into compile time.
2022-04-03 17:58:14 +00:00
zugz (tox)
e49a477a84
feat: add forwarding and announce request handling
This is the "server-side" part of the new friend finding system,
allowing DHT nodes to store small amounts of data and permit searching
for it. A forwarding (proxying) mechanism allows this to be used by TCP
clients, and deals with non-transitivity in the network.
2022-04-03 00:00:08 +00:00
iphydf
c71b1218f8
chore(deps): Use upstream cmp directly instead of our fork.
The upstream maintainer is active and has merged our changes.
2022-04-03 09:02:31 +00:00
iphydf
e3ace8ca24
cleanup: Make *_free functions nullable.
These should be no-ops when passed a null pointer.
2022-04-02 21:28:26 +00:00
iphydf
eb07575334
cleanup: Remove layers in the cmake build.
Nothing checks whether these layers are actually observed. The bazel
build does check this, so there's no need to have this documentation in
the cmake build. It'll just go out of date.
2022-04-01 17:17:15 +00:00
iphydf
cc31ff07fa
feat: Add support for custom random number generator.
This can be used by fuzzers to make RNG-driven code deterministic (i.e.
based on the fuzzer input).
2022-04-01 14:06:05 +00:00
iphydf
660e346fce
cleanup: Disallow stack frames of over 9000 bytes.
This only happens in tests, which are easy to fix. Inside toxcore we
should actually be more stringent, but for now this helps already.
2022-04-01 13:57:16 +00:00
iphydf
d0ebc21a0e
cleanup: Add Network object parameter for addr_resolve.
This function doesn't use Network yet, but it will in the future, and
for now it's better to pass Network to all network I/O functions.
2022-04-01 10:42:43 +00:00
iphydf
e04484efae
cleanup: Remove unused random_testing program.
Fuzzers do a better job of this.
2022-04-01 09:20:04 +00:00
iphydf
b09a1ff02d
cleanup: Remove all uses of TOX_*_MAX_SIZE macros.
We should use the functions for these instead.
2022-04-01 09:10:37 +00:00
jfreegman
015305a088
Merge moderation portion of new groupchats codebase 2022-03-30 20:09:50 -04:00
iphydf
09575dc05b
chore: Clone submodules in clusterfuzzlite Dockerfile.
Needed for the build afterwards.

Also added the cflite Dockerfile to automatic CI builds so changes to it
are checked in pull requests.

Also fixed the tokstyle docker image. It needs clang instead of gcc now.
2022-03-30 18:29:46 +00:00
iphydf
f39aac09e6
feat: Allow custom network functions.
The idea here is to have a `Network` object that contains functions for
network operations and an optional userdata object that can manage those
network operations. This allows e.g. a fuzzer to replace the network
functions with no-ops or fuzzer inputs, reducing the need for `#ifdef`s.
2022-03-29 22:03:26 +00:00
iphydf
24065cf5a9
chore: Update bootstrap daemon checksum. 2022-03-27 23:30:37 +00:00
iphydf
b768c153ae
chore: Add third_party to toxcore-sources docker image.
This is needed so other/docker/coverage/run works.
2022-03-27 22:01:19 +00:00
iphydf
bb7b1e9b3a
cleanup: Make addr_resolve a private function.
This isn't used anywhere except in network_test. That test now checks
behaviour of the function actually used elsewhere in tox code, instead.
2022-03-27 20:10:42 +00:00
sudden6
bc006beb4e
cleanup: split CMakeLists.txt and add some missing targets 2022-03-27 20:09:10 +02:00
iphydf
ac3e8fea63
test: Improve stability of tox_many_tcp_test.
If the TCP port is in use, try the next TCP port.
2022-03-27 15:10:24 +00:00
iphydf
ee154da4a9
refactor: Use cmp instead of msgpack-c for events packing.
This library is better in every way.
2022-03-26 23:57:15 +00:00
iphydf
6749974470
cleanup: Expose struct Tox to internal code.
To avoid unsafe casts that tokstyle doesn't allow and that can cause
random breakages.
2022-03-26 21:15:31 +00:00
iphydf
1928704065
test: Add fuzzer tests to the bazel build. 2022-03-25 18:48:41 +00:00
iphydf
2dcb946e62
cleanup: Reduce scope of array-typed variables where possible.
Cimple cannot actually find these without also causing false positives,
but I found them with cimple before removing the code causing false
positives again.
2022-03-21 12:02:46 +00:00
iphydf
1fc05ce616
cleanup: Reduce scope of variables as requested by cimple. 2022-03-20 23:24:09 +00:00
jfreegman
26c2bb1061
Merge onion_announce changes from new groupchats fork
These changes will make the DHT compatible with the new groupchat
implementation. Additionally, onion announces now support arbitrary
extra data
2022-03-20 17:33:06 -04:00
iphydf
c3a938e38c
cleanup: Avoid memset on structs.
Most of these were safe (some were not), but even the safe ones can
become unsafe (non-portable) later on when adding pointer or float members.
2022-03-19 12:32:36 +00:00
iphydf
88ae190aca
cleanup: Split the huge TCP client packet handler.
Also enable cppcheck in both C and C++ mode.
2022-03-19 11:06:50 +00:00
iphydf
2b2eb41211
chore: Enable memory sanitizer build.
I don't know if this will actually work, or how many of these "fixes" I
need to get msan to be happy on CI. For me locally, it all works fine.
On CI, for some reason it's not fine even though I run in the exact same
docker image as CI.
2022-03-19 03:11:16 +00:00
iphydf
864b4aa01f
chore: Be even more explicit about integer range bounds.
For coverity, which continues to think we're overrunning buffers when
at this point it's easy to prove we're not. Here would be the corrected
coverity finding:

6. Condition packet_length <= 105 /* 1 + 32 * 2 + 24 + 16 */, taking false branch.
7. Condition packet_length > 1024, taking false branch.

Now packet_length must be > 105 and <= 1024.

12. Condition len1 == packet_length - (89 /* 1 + 32 * 2 + 24 */) - 16, taking true branch.

len1 must be > 0 (105 - 89 - 16) and <= 919 (1024 - 89 - 16).

14. decr: Decrementing len1. The value of len1 is now between 0 and 919 (inclusive).

This is where coverity goes wrong: it thinks len1 could be up to 2147483629.

15. buffer access should be OK. Coverity thinks it's not.
2022-03-08 20:46:46 +00:00
iphydf
001aa63401
fix: Allow port range in DHT_bootstrap.c. 2022-03-08 12:16:08 +00:00
iphydf
d15d9fa72d
cleanup: Minor cleanups in TCP_connection.c.
Nothing very noteworthy, I just came across this and made it slightly
more readable.

I'm not making this function `bool` right now because it's used in NGC
and that will break.
2022-03-08 07:53:45 +00:00
iphydf
9e26185678
cleanup: Disable LAN discovery in TCP-only mode.
It causes warnings in the logs.

Also added the IP/Port to TCP connect logging.
2022-03-07 00:10:49 +00:00
iphydf
0d8b9b6b3d
cleanup: Add assertion for decrypted data length.
Hopefully this will make it clear enough to coverity that we're not
overrunning any buffers.
2022-03-06 18:06:39 +00:00
Robin Linden
0467101381
Release 0.2.17 2022-03-06 01:45:06 +01:00
iphydf
e1e92c1ac1
docs: Clean up doxygen comments to be more doxygen-like. 2022-03-05 22:30:48 +00:00