Commit Graph

700 Commits

Author SHA1 Message Date
iphydf
941026266e
refactor: Allow overriding mono_time in tox_new.
This makes it so if mono_time is overridden, no monotonic time-related
system call is invoked in tox_new.
2022-04-03 22:48:16 +00:00
Robin Linden
657d185de5
chore: Support producing shared libraries on Windows
The FLAT_OUTPUT_STRUCTURE CMake option was added to allow the tests to
find the .dlls.

The global data symbols were converted to functions to allow
WINDOWS_EXPORT_ALL_SYMBOLS handle them.

See:
https://cmake.org/cmake/help/v3.23/prop_tgt/WINDOWS_EXPORT_ALL_SYMBOLS.html
2022-04-03 23:04:11 +02:00
iphydf
ddda605509
fix: Don't crash if RNG init failed.
`system_random()` can fail and return NULL, which should be handled by
toxencryptsave functions.

Also synced function comments between .h and .c file for toxencryptsave.
2022-04-03 20:46:44 +00:00
zugz (tox)
4430515052
cleanup: remove populate_path_nodes_tcp
It was a no-op.
2022-04-03 00:00:00 +00:00
iphydf
7a3ead591f
cleanup: Use a struct for the ip_ntoa buffer.
Every use of this function needs to allocate the same buffer. None of
the callers uses a differently sized buffer, so we might as well put it
in a struct and have the type checker prove the buffer size is correct.

Also rename `ip_ntoa` to `net_ip_ntoa` to avoid clashes with ESP-IDF
system libraries which define this function as well.
2022-04-03 18:57:04 +00:00
iphydf
e4d1958ffa
refactor: Allow NULL logger; make it no-op in NDEBUG. 2022-04-03 18:29:22 +00:00
iphydf
378febffff
cleanup: Use static_assert instead of assert where possible.
This avoids some "always true condition" warnings and lifts the errors
(if any) into compile time.
2022-04-03 17:58:14 +00:00
zugz (tox)
e49a477a84
feat: add forwarding and announce request handling
This is the "server-side" part of the new friend finding system,
allowing DHT nodes to store small amounts of data and permit searching
for it. A forwarding (proxying) mechanism allows this to be used by TCP
clients, and deals with non-transitivity in the network.
2022-04-03 00:00:08 +00:00
iphydf
c71b1218f8
chore(deps): Use upstream cmp directly instead of our fork.
The upstream maintainer is active and has merged our changes.
2022-04-03 09:02:31 +00:00
iphydf
e3ace8ca24
cleanup: Make *_free functions nullable.
These should be no-ops when passed a null pointer.
2022-04-02 21:28:26 +00:00
iphydf
eb07575334
cleanup: Remove layers in the cmake build.
Nothing checks whether these layers are actually observed. The bazel
build does check this, so there's no need to have this documentation in
the cmake build. It'll just go out of date.
2022-04-01 17:17:15 +00:00
iphydf
cc31ff07fa
feat: Add support for custom random number generator.
This can be used by fuzzers to make RNG-driven code deterministic (i.e.
based on the fuzzer input).
2022-04-01 14:06:05 +00:00
iphydf
660e346fce
cleanup: Disallow stack frames of over 9000 bytes.
This only happens in tests, which are easy to fix. Inside toxcore we
should actually be more stringent, but for now this helps already.
2022-04-01 13:57:16 +00:00
iphydf
d0ebc21a0e
cleanup: Add Network object parameter for addr_resolve.
This function doesn't use Network yet, but it will in the future, and
for now it's better to pass Network to all network I/O functions.
2022-04-01 10:42:43 +00:00
iphydf
e04484efae
cleanup: Remove unused random_testing program.
Fuzzers do a better job of this.
2022-04-01 09:20:04 +00:00
iphydf
b09a1ff02d
cleanup: Remove all uses of TOX_*_MAX_SIZE macros.
We should use the functions for these instead.
2022-04-01 09:10:37 +00:00
jfreegman
015305a088
Merge moderation portion of new groupchats codebase 2022-03-30 20:09:50 -04:00
iphydf
09575dc05b
chore: Clone submodules in clusterfuzzlite Dockerfile.
Needed for the build afterwards.

Also added the cflite Dockerfile to automatic CI builds so changes to it
are checked in pull requests.

Also fixed the tokstyle docker image. It needs clang instead of gcc now.
2022-03-30 18:29:46 +00:00
iphydf
f39aac09e6
feat: Allow custom network functions.
The idea here is to have a `Network` object that contains functions for
network operations and an optional userdata object that can manage those
network operations. This allows e.g. a fuzzer to replace the network
functions with no-ops or fuzzer inputs, reducing the need for `#ifdef`s.
2022-03-29 22:03:26 +00:00
iphydf
24065cf5a9
chore: Update bootstrap daemon checksum. 2022-03-27 23:30:37 +00:00
iphydf
b768c153ae
chore: Add third_party to toxcore-sources docker image.
This is needed so other/docker/coverage/run works.
2022-03-27 22:01:19 +00:00
iphydf
bb7b1e9b3a
cleanup: Make addr_resolve a private function.
This isn't used anywhere except in network_test. That test now checks
behaviour of the function actually used elsewhere in tox code, instead.
2022-03-27 20:10:42 +00:00
sudden6
bc006beb4e
cleanup: split CMakeLists.txt and add some missing targets 2022-03-27 20:09:10 +02:00
iphydf
ac3e8fea63
test: Improve stability of tox_many_tcp_test.
If the TCP port is in use, try the next TCP port.
2022-03-27 15:10:24 +00:00
iphydf
ee154da4a9
refactor: Use cmp instead of msgpack-c for events packing.
This library is better in every way.
2022-03-26 23:57:15 +00:00
iphydf
6749974470
cleanup: Expose struct Tox to internal code.
To avoid unsafe casts that tokstyle doesn't allow and that can cause
random breakages.
2022-03-26 21:15:31 +00:00
iphydf
1928704065
test: Add fuzzer tests to the bazel build. 2022-03-25 18:48:41 +00:00
iphydf
2dcb946e62
cleanup: Reduce scope of array-typed variables where possible.
Cimple cannot actually find these without also causing false positives,
but I found them with cimple before removing the code causing false
positives again.
2022-03-21 12:02:46 +00:00
iphydf
1fc05ce616
cleanup: Reduce scope of variables as requested by cimple. 2022-03-20 23:24:09 +00:00
jfreegman
26c2bb1061
Merge onion_announce changes from new groupchats fork
These changes will make the DHT compatible with the new groupchat
implementation. Additionally, onion announces now support arbitrary
extra data
2022-03-20 17:33:06 -04:00
iphydf
c3a938e38c
cleanup: Avoid memset on structs.
Most of these were safe (some were not), but even the safe ones can
become unsafe (non-portable) later on when adding pointer or float members.
2022-03-19 12:32:36 +00:00
iphydf
88ae190aca
cleanup: Split the huge TCP client packet handler.
Also enable cppcheck in both C and C++ mode.
2022-03-19 11:06:50 +00:00
iphydf
2b2eb41211
chore: Enable memory sanitizer build.
I don't know if this will actually work, or how many of these "fixes" I
need to get msan to be happy on CI. For me locally, it all works fine.
On CI, for some reason it's not fine even though I run in the exact same
docker image as CI.
2022-03-19 03:11:16 +00:00
iphydf
864b4aa01f
chore: Be even more explicit about integer range bounds.
For coverity, which continues to think we're overrunning buffers when
at this point it's easy to prove we're not. Here would be the corrected
coverity finding:

6. Condition packet_length <= 105 /* 1 + 32 * 2 + 24 + 16 */, taking false branch.
7. Condition packet_length > 1024, taking false branch.

Now packet_length must be > 105 and <= 1024.

12. Condition len1 == packet_length - (89 /* 1 + 32 * 2 + 24 */) - 16, taking true branch.

len1 must be > 0 (105 - 89 - 16) and <= 919 (1024 - 89 - 16).

14. decr: Decrementing len1. The value of len1 is now between 0 and 919 (inclusive).

This is where coverity goes wrong: it thinks len1 could be up to 2147483629.

15. buffer access should be OK. Coverity thinks it's not.
2022-03-08 20:46:46 +00:00
iphydf
001aa63401
fix: Allow port range in DHT_bootstrap.c. 2022-03-08 12:16:08 +00:00
iphydf
d15d9fa72d
cleanup: Minor cleanups in TCP_connection.c.
Nothing very noteworthy, I just came across this and made it slightly
more readable.

I'm not making this function `bool` right now because it's used in NGC
and that will break.
2022-03-08 07:53:45 +00:00
iphydf
9e26185678
cleanup: Disable LAN discovery in TCP-only mode.
It causes warnings in the logs.

Also added the IP/Port to TCP connect logging.
2022-03-07 00:10:49 +00:00
iphydf
0d8b9b6b3d
cleanup: Add assertion for decrypted data length.
Hopefully this will make it clear enough to coverity that we're not
overrunning any buffers.
2022-03-06 18:06:39 +00:00
Robin Linden
0467101381
Release 0.2.17 2022-03-06 01:45:06 +01:00
iphydf
e1e92c1ac1
docs: Clean up doxygen comments to be more doxygen-like. 2022-03-05 22:30:48 +00:00
iphydf
3275bb8fea
cleanup: Add explicit callback setters for MSI callbacks.
Instead of a callback ID and a `switch`.
2022-03-04 10:33:23 +00:00
iphydf
cff9a18e39
cleanup: Split util.c out of the network library.
Also, don't include ccompat.h in header files. Instead, explicitly
include it in .c files.
2022-03-04 10:33:11 +00:00
iphydf
f19524cb21
cleanup: Remove dependency from LAN_discovery onto DHT.
As a side-effect, DHT now always accepts LAN discovery packets, even
when LAN discovery is disabled. When LAN discovery is disabled, those
packets are ignored.
2022-03-04 10:19:05 +00:00
iphydf
21688d1d34
cleanup: Move test-only functions into tests.
Also inlined `new_networking` in most places.
2022-03-03 23:58:43 +00:00
iphydf
2cc8dafd4c
cleanup: Remove redundant Messenger and DHT tests.
These don't test anything that isn't covered by higher level tox tests.
These are also not unit tests and have never found any bug that wasn't
also caught by other tests. This makes them a pure maintenance burden.
2022-03-03 21:17:03 +00:00
iphydf
ad2ed9e837
test: Add unit test for create/handle request packets.
Also added an assert to tell coverity that we don't actually have an
overflow (or underflow) of the length parameter.
2022-03-03 18:00:17 +00:00
iphydf
6c2014d719
cleanup: Remove port from Broadcast_Info.
This is added later when we actually send the broadcasts.
2022-03-03 15:21:34 +00:00
iphydf
fe4da6a541
cleanup: Log at ERROR level when connect() fails.
Failure is defined as either being passed an invalid IP/Port, or some
unexpected error code.
2022-02-28 14:01:01 +00:00
iphydf
3ef96f7096
cleanup: Remove *all* boolean-int conversions.
These were found by the new cimple type check which is completely
unforgiving to implicit boolean conversions. After this PR, there should
be no more implicit int-to-bool or bool-to-int conversions.
2022-02-27 00:29:43 +00:00
iphydf
163e9c3b03
cleanup: Remove more boolean conversions (and a bugfix).
These were found by the new stronger type check in cimple. The one
bugfix is in `crypto_sha512_cmp`, which used to think `crypto_verify_32`
returns bool while actually it's -1/0/1.
2022-02-26 00:03:56 +00:00
iphydf
093927ba4f
test: Add mallocfail and proxy test to our coverage runs. 2022-02-25 01:55:28 +00:00
iphydf
aa72c47125
test: Add test coverage docker build for local tests.
This uses mallocfail to further increase coverage using the existing
tests. Also:
* Moved the non-auto "tox_one_test" to gtest. This should be
  split into smaller tests later.
* Changed `hole_punching` to `bool`.
2022-02-24 16:24:28 +00:00
iphydf
2ce6f5b169
fix: Report failure to DHT bootstrap back to the client.
Also reduce log verbosity a bit.
2022-02-24 11:37:47 +00:00
Robin Linden
0ac930e1f6
cleanup: Clean up comparing bools with ints 2022-02-23 22:50:12 +00:00
sudden6
3e6fc59f91
chore: remove cpufeatures.c
The method to detect CPU features in Android has changed and this is no
longer needed.
2022-02-23 20:59:06 +01:00
iphydf
b13590a86d
cleanup: Mark all local non-pointers as const where possible. 2022-02-23 19:30:01 +00:00
iphydf
17c8f50f44
cleanup: Reduce name shadowing; remove ptr-to-bool conversions.
Missed a few of those in check-c. check-cimple now catches these with a
stronger type check.

Other changes:
* `ptr + int` must always have the `ptr` first, so `int + ptr` is not
  allowed anymore.
* `close` and `time` were shadowing libc functions. `file_data` was
  shadowed in a function (and is not a good function name anyway), so
  renamed to `send_file_data` which is more descriptive.
* Within a function, all local variables of the same name must have the
  same type.
* The `strerror_r` change wasn't necessary, but I kept it because it
  seems a bit clearer to me now. `#ifdef`s inside functions are a bit
  confusing sometimes.
2022-02-23 17:30:53 +00:00
iphydf
6be655c56c
cleanup: Return boolean constants, not ints from bool functions. 2022-02-22 17:50:17 +00:00
iphydf
b7f95bbe70
cleanup: Remove any disallowed casts.
The ones in toxav weren't needed. The ones in network.c are
non-trivially correct. Now, the code is more easily verifiable.
2022-02-22 02:19:29 +00:00
iphydf
e7bc611dac
cleanup: Enable most cppcheck warnings as errors.
Cleaned up some of the warnings it gives. Disabled the ones that are
less useful for us at this time.
2022-02-21 17:17:36 +00:00
iphydf
a9ccf1be26
cleanup: Remove all implicit bool conversions in if-conditions.
Note to reviewer: please review carefully. This was done manually.
2022-02-21 15:10:22 +00:00
iphydf
6a6bc029de
cleanup: Remove unused execution trace library. 2022-02-19 11:44:13 +00:00
Robin Linden
1937ee39c3
Release 0.2.16 2022-02-19 00:51:01 +01:00
iphydf
c5e3bca6de
chore: Simplify and speed up nacl build using toxchat/nacl. 2022-02-18 22:13:42 +00:00
iphydf
4389c71619
test: Add a profiling script and Dockerfile.
This works on my kernel. If your kernel is older/newer, you may need a
different alpine base image that matches your kernel more closely.
2022-02-18 21:38:55 +00:00
iphydf
84a03bc2ed
cleanup: Enable tokstyle's -Wlarge-struct-params. 2022-02-18 10:49:46 +00:00
iphydf
cc045d6343
cleanup: Avoid implicit boolean and floating point conversions in decls.
Also avoid implicit enum-to-int.
2022-02-18 03:26:12 +00:00
jfreegman
0f3534656f
Improve how we share TCP relays with friends
Previously we would try to send three random TCP relays that we're
connected to to each friend once every 5 minutes. The problem with
this method is that it could take an extraordinarily long time
to share every relay; some relays might be consistently skipped
while others might be sent repeatedly. Moreover, there's no
guarantee that the nodes you try to send are actually online.
This leads to a prety unreliable and flaky way of sharing.

Now we reduce the timer to two minutes, and cycle through the list
trying 3 nodes each share attempt. This guarantees that every online
node in our list gets shared with every friend after a fixed amount of
time (which depends on how many nodes are in the list)
2022-02-17 16:16:29 -05:00
iphydf
5e724d8909
cleanup: Avoid implicit pointer-to-bool conversion in if in toxav. 2022-02-17 20:33:24 +00:00
iphydf
933c6b7517
refactor: Compare pointers in if conditions to nullptr.
Don't use implicit ptr-to-bool conversion.
2022-02-17 19:55:29 +00:00
iphydf
43fc2374bc
refactor: Add a bit more logging; change WARNING to ERROR.
ERROR is caught in tests, so we'll use it more for things that ought not
to happen.
2022-02-16 23:05:10 +00:00
iphydf
1d0f6e51e2
cleanup: Add some more error path logging to TCP server code. 2022-02-15 12:08:22 +00:00
iphydf
c9d3e078e2
chore: Add BUILD file for websockify. 2022-02-14 13:56:18 +00:00
Maxim Biro
dcc65c385f
Update Windows Docker build deps 2022-02-13 22:54:08 -05:00
iphydf
a3cd1102f7
perf: Reduce minimal encoding size of packed events.
We're using a union-like encoding now with an enum telling which union
member to set.
2022-02-12 17:44:44 +00:00
iphydf
33588c1aad
cleanup: Don't use VLAs for huge allocations.
One of these was creating a single 262144 byte stack frame. We now have
a way to check and limit the allocation size of a VLA. The `Cmp_Data`
ones were also fairly large. Now, no allocation is larger than 2KiB
(though rtp.c allocates close to that much).
2022-02-11 03:05:30 +00:00
jfreegman
73ba0504b0
Add autotest for dht getnodes API
The test creates a small DHT and makes sure that each
peer is able to crawl the full network
2022-02-10 10:33:34 -05:00
jfreegman
652ae27eeb
Add DHT queries to private API
This commit adds functionality for clients to interact with
the DHT, sending getnodes requests to their peers and receiving
nodes in getnodes responses.
2022-02-10 10:33:31 -05:00
Maxim Biro
2965403e2c
Add bash-completion for tox-bootstrapd 2022-02-10 02:40:42 -05:00
iphydf
10f86f6c00
cleanup: Add more nonnull and nullable annotations. 2022-02-09 20:03:34 +00:00
iphydf
5a3a0b6453
cleanup: Add nonnull annotations to function definitions.
These help static analysis and ubsan. We should eventually have all
functions annotated like this with a cimple check to make sure every
pointer has an explicit nullability annotation. The `nullable`
annotation does nothing in GCC, but will be used by cimple to validate
that every parameter has defined nullability.
2022-02-08 17:57:47 +00:00
iphydf
6f36b67a13
chore: Add an undefined behaviour/integer sanitizer build. 2022-02-08 09:18:18 +00:00
iphydf
0a2190f28c
feat: Add binary packing functions for tox events. 2022-02-07 02:43:24 +00:00
iphydf
9218566599
cleanup: Make LAN discovery thread-safe without data races.
It was kind of thread-safe, maybe, but there was a data race that makes
tsan unhappy. We now do interface detection once per Tox instance
instead of once per process.
2022-02-06 19:56:34 +00:00
iphydf
de4af4c270
feat: Add async event handling (callbacks) code.
Instead of synchronously handling events as they happen in
`tox_iterate`, this first collects all events in a structure and then
lets the client process them. This allows clients to process events in
parallel, since the data structure returned is mostly immutable.

This also makes toxcore compatible with languages that don't (easily)
support callbacks from C into the non-C language.

If we remove the callbacks, this allows us to add fields to the events
without breaking the API.
2022-02-06 17:28:28 +00:00
iphydf
cde796f1f8
cleanup: Comply with new cimple callback rules.
* Function arguments must use `foo_cb *p` and can't just use `foo_cb p`
* You can no longer cast function pointers (if it's incompatible, you
  must wrap the callback). I'm avoiding this with tokstyle exclusions.
2022-02-06 16:45:41 +00:00
iphydf
4d4120214a
chore: Add an easy way to run autotools and circleci builds locally. 2022-02-06 15:48:02 +00:00
Robin Linden
d93e795cfe
Release 0.2.15 2022-02-06 04:40:03 +01:00
iphydf
79eb17aeac
fix: Reduce logging verbosity even more. 2022-02-06 03:33:34 +00:00
iphydf
fedcdea95a
fix: Reduce logging verbosity in TCP server. 2022-02-06 03:06:27 +00:00
Robin Linden
f41caa2a1e
Release 0.2.14 2022-02-05 22:24:58 +01:00
iphydf
d39f803c7e
docs: Make crypto_core.h appear on doxygen. 2022-02-05 19:34:13 +00:00
jfreegman
96814c4645
Use proper method for generating random numbers in a range
Using modulus creates a bias in its output. The libsodium
randombytes_uniform() function guarantees a uniform distribution
of possible outputs
2022-02-05 11:14:45 -05:00
iphydf
2ed9b1927f
doc: Add doxygen configuration and netlify publishing. 2022-02-05 15:59:45 +00:00
iphydf
1859d0f44a
cleanup: Ensure we limit the system headers included in .h files.
Most system headers contain functions (e.g. `memcpy` in `string.h`)
which aren't needed in our own header files. For the most part, our own
headers should only include types needed to declare our own types and
functions. We now enforce this so we think twice about which headers we
really need in the .h files.
2022-02-04 20:54:37 +00:00
Maxim Biro
cda6c9b6e8
Fix potential freeing of an immutable static buffer
strerror_r() has two versions: GNU-specific and XSI-compliant. The XSI
version always stores the string in the provided buffer, but the GNU
version might store it in the provided buffer or it might use some
immutable static buffer instead. Since we always free the error string,
we might end up freeing the immutable static buffer.
2022-02-04 08:09:14 -05:00
iphydf
28dc8c1ded
cleanup: Don't pass the whole DHT object to lan discovery.
It only needs net and dht public key.
2022-02-03 20:50:36 +00:00
iphydf
014cc42249
chore: Make run-infer script use docker.
So people can run it locally more easily.
2022-02-03 20:11:16 +00:00
jfreegman
1d3778fc53
Move duplicate code for converting IPv4-in-6 to IPv4 to a function 2022-02-03 12:32:30 -05:00
jfreegman
5619b5e056
Make more functions take const pointers to IP_Port
We additionally now make local copies of the IP_Port param instead
of modifying the passed argument
2022-02-03 11:17:29 -05:00