The default stack size for musl-libc is 128kb. Therefore we should try to keep stack
allocations well below this limit in order to avoid stack overflows.
We no longer allow `int a, b;`. In the few cases where we used it, we
instead better
* limit the scope of the identifier (e.g. in a for-init-decl)
* split the line and have 2 separate declarators, because the
identifiers designate different types of things (e.g. friend numbers
and group numbers).
This check puts all of our code in a C++ anonymous namespace, which is
effectively making all functions `static`. This allows the compiler to
determine that a function is unused, so we can delete it.
* Use-after-free because we free network before dht in one case.
* Various unchecked allocs in tests (not so important).
* We used to not check whether ping arrays were actually allocated in DHT.
* `ping_kill` and `ping_array_kill` used to crash when passing NULL.
Also:
* Added an assert in all public API functions to ensure tox isn't NULL.
The error message you get from that is a bit nicer than "Segmentation
fault" when clients (or our tests) do things wrong.
* Decreased the sleep time in iterate_all_wait from 20ms to 5ms.
Everything seems to still work with 5ms, and this greatly decreases
the amount of time spent per test run, making oomer run much faster.
* Use fully static build for the bootstrap daemon.
* Store a sha256sum of the binary in the repo.
* Updated documentation for it.
* Add support for fully static build in cmake.
* Enable the docker build on every PR, so we catch changes to the
checksum. I realise this is adding toil, but having the checksum is
valuable for security of released binaries.
This one is unfortunately a little more complicated to use. I may add a
simpler API later, but for now, it's JSON-based (because I couldn't get
binary data to work without it getting mangled somewhere along the way -
JSON is at least text-transport-safe).
tox-bootstrapd can use around 600 TCP sockets during TCP server's normal
functioning. Many systems default to having a soft limit of 1024 open file
descriptors, which we are close to reaching, so it was suggested we bump that
limit to a higher number. iphy suggested increasing it to 32768.
We should avoid recursion, as it makes reasoning about stack growth
harder. This tool shows (currently) 4 (non-tail) recursive functions, at
least 2 of which are easy to fix.
Don't know why codes with macro dosen't work.
As it's only a few expensive, just code it without macro for now.
\#if (MIN_LOGGER_LEVEL == LOG_TRACE) || (MIN_LOGGER_LEVEL == LOG_DEBUG)
fprintf(stderr, "[%s] %s:%d(%s) %s\n", strlevel, file, line, func, message);
\#endif
Reduced by, e.g.:
* `file_transfer_test`: 33% of the `clock_gettime` calls.
* `tox_many_test`: 53% of the `clock_gettime` calls.
Other tests will see similar improvements. Real world applications will
be closer to 40-50% improvement, since tox_many_test has 100 nodes, while
file_transfer_test has 2 nodes.
This allows Tox to contain additional data on top of Messenger, making
Messenger not necessarily the most top-level object. E.g. groups are
built on Messenger and currently awkwardly void-pointered into it to
pretend there is no cyclic dependency.
It turns out, `unix_time` is also monotonic, and is used as such, so I've
renamed the new functions to `mono_time_*`.
2018-07-08:
```
00:01 <@irungentoo> the idea used to be that the unix_time() function
could go backward in time but I think I might have started using it like
if it could not after I changed it so that it would never go back in time
```
Rules:
1. Constants are uppercase names: THE_CONSTANT.
2. SUE[1] types start with an uppercase letter and have at least one
lowercase letter in it: The_Type, THE_Type.
3. Function types end in "_cb": tox_friend_connection_cb.
4. Variable and function names are all lowercase: the_function.
This makes it easier for humans reading the code to determine what an
identifier means. I'm not convinced by the enum type name change, but I
don't know a better rule. Currently, a lot of enum types are spelled like
constants, which is confusing.
[1] struct/union/enum
This is needed for libvpx to work on android.
This also means that we can upload our test binaries to an android device
and actually run them, now that libcheck is no longer a blocker.
The first round some tests will fail and others will segfault. The second
round, still some will fail, but the third round it might pass. At some
point, tests will pass.
The reason for this is that tests assume a lot about which ports they are
given, and also toxcore's default port range has too few ports to run all
tests in parallel. These issues will be fixed in future PRs.
Also, add an auto-test for bootstrap and for LAN discovery.
Bootstrap is never tested otherwise, and LAN discovery is a prerequisite
for everything else. Having these two tests lets us rule out or identify
LAN discovery as a possible cause for test failures.
Also reduce number of people in conference to 5, because on Circle CI the
test times out trying to connect more than 6 or 7 people. The persistent
conferences PR will improve this so we can set it much higher then.
Since trusty has more up-to-date packages, we can remove some of the
custom install code. Also, we're not using hstox at the moment, so there
is no need to slow down the builds for now.
This allows us and users to reproducibly build verified versions of the
library with checksums. It will power the toktok-stack continuous build
with checked-in checksums at specific git revisions.
To prevent top-level linking of all libraries.
Problem: ```pkg-config --libs toxcore``` returns all libraries that are
required by all libtox*.so libraries. This is wrong because for a
dynamically linked executable only top-level libraries need to be
supplied. ```pkg-config --libs --static toxcore``` should return all
libraries for the statically linked executable.
For example, the ToxBot https://github.com/JFreegman/ToxBot executable
uses pkg-config and is linked with the opus library, which is wrong.
Based on #533.
This basically means: try until you run out of time (50 minutes on
Travis). On Linux, we really want the tests to pass, so there is no point
in limiting the number of retries. On windows, we don't retry, on FreeBSD
it's limited to 1.
This library contains all the code for the old libtoxcore, libtoxav,
libtoxdns, and libtoxencryptsave. The build for toxav is still optional,
and disabling it causes libtoxcore to simply not contain those symbols
and the pkg-config file to not include opus and vpx as dependencies.
This requires that every symbol, even if static (file-scope), is unique.
The idea is that we can easily run "whole" program static analysis on
programs that include monolith.h ("whole" is in quotes, as we don't
include dependencies like libsodium in this static analysis).
"All rights reserved" was incorrect. The project was licensed under GPL3,
which means a lot of rights are licensed to everybody in the world, i.e.
not reserved to the "Tox Project".
this updates the version-sync script to generate proper SO versions
which will be used by cmake and libtool to create version symlinks
on the system when a library is installed as well as setting the SO
version in the binary.
To see what this does, you have to configure tox with a prefix:
./configure --prefix=/tmp/tox-with-libtool
mkdir cbuild && cd cbuild && cmake -DCMAKE_INSTALL_PREFIX=/tmp/tox-with-cmake ..
Then run `make && make install`.
in both instances you should see the following installed in `lib/`:
libtoxcore.so -> libtoxcore.so.1.4.0
libtoxcore.so.1 -> libtoxcore.so.1.4.0
libtoxcore.so.1.4.0
inside the binary the soname should be the one with .1 and it should not
contain the full version:
$ objdump -p libtoxcore.so.1.4.0 | grep SONAME
SONAME libtoxcore.so.1
- CFLAG gnu99 was changed to c99.
- CXXFLAG c++98 was changed to c++11.
- CFLAG -pedantic-errors was added so that non-ISO C now throws errors.
- _XOPEN_SOURCE feature test macro added and set to 600 to expose SUSv3
and c99 definitions in modules that required them.
- Fixed tests (and bootstrap daemon logging) that were failing due to
the altered build flags.
- Avoid string suffix misinterpretation; explicit narrowing conversion.
- Misc. additions to .gitignore to make sure build artifacts don't wind
up in version control.
These were generated by the autotools build. Some clients may depend on
these files instead of the newer split pkg-config files. New clients
should be using the toxcore, toxav, toxencryptsave, and toxdns modules.
We also never really tested this, because we run make distcheck, which
does another configure with default flags instead of the ones we passed.
Fixes#317.
Having -Werror set by default causes users' builds to fail because
toxcore is not warning-free. Failing on errors is appropriate for the
development phase, e.g. when building it in a CI enviroment, but it
doesn't make much sense to fail builds for users and let them figure out
that they need to pass -DWARNINGS=OFF to make the library build.
Compiling as C++ changes nothing semantically, but ensures that we don't
break C++ compatibility while also retaining C compatibility.
C++ compatibility is useful for tooling and additional diagnostics and
analyses.
We use TRACE=ON (cmake flag) to enable LOG_TRACE. This way, a regular
build can enable DEBUG while not paying the price of TRACE. This is
particularly important for FFI bindings (especially Python), where
invoking callbacks can be an expensive operation.
When cross-compiling to <target> from Linux, cmake might find native Linux
libconfig and decide to build tox-bootstrapd. If the target is Windows, this
will fail, as tox-bootstrapd can't be built for Windows in the first place. If
the target is Linux of some other architecture, then using host native
libconfig will fail too. Thus an option is needed to guard against this.
- Moved apidsl headers next to their generated versions. In the future,
perhaps all (or most) headers will be apidsl-generated, so the sources
should stay together.
- Try to find apidsl/apigen binary and astyle binary and use it for the
format test. Don't run the format test if these can't be found.
- Fixed incorrect parameter names (documented name didn't match code
name).
- Removed `@return` from functions that return `void`.
- Make sure every parameter is documented. This required moving the
planes and strides documentation to the function docs.
It is still C code, so still compatible with C compilers as well. This
change lets us see more clearly where implicit conversions occur by
making them explicit.
- All global variables should be static unless they have an explicit
extern declaration in a header file.
- `to_compare` was not used in encryptsave and toxav tests.
- `break` in switch cases is not required directly after `return`,
`goto`, or a noreturn function like `abort`.
Enabling this flag will generate and use an LD version script. It
ensures that the dynamic libraries (libtoxcore.so, libtoxav.so) only
export the symbols that are defined in their public API (tox.h and
toxav.h, respectively).
In the future, all TODOs added either need a bug number (TODO(#NN)) or a
person's github user name. By default, I made irungentoo the owner of
all toxcore TODOs, mannol the owner of toxav TODOs, and myself the owner
of API TODOs.
This test is already performed by `make test` later on. We originally
had it in the Travis script to make it fail fast when the format is
wrong, but there is also some value in running all tests despite format
errors.
Fixes#83. There are no more relevant phases that would benefit from the
padding lines proposed.
It now enforces a bit more formatting. In particular, padding inside
parentheses is removed. I would like it to remove padding after unary
operators, but there seems to be no option for that.
I hadn't done this for the "fun" code, yet. Also, we should include
system headers after our own headers.
"In general, a module should be implemented by one or more .cpp files.
Each of these .cpp files should include the header that defines their
interface first. This ensures that all of the dependences of the module
header have been properly added to the module header itself, and are not
implicit. System headers should be included after user headers for a
translation unit."
-- http://llvm.org/docs/CodingStandards.html#a-public-header-file-is-a-module
- Any non-externally-visible declarations should be `static`.
- Casting away the `const` qualifier from pointers-to-const is
dangerous. All but one instance of this are now correct. The one
instance where we can't keep `const` is one where toxav code actually
writes to a chunk of memory marked as `const`. This code also assumes
4 byte alignment of data packets. I don't know whether that is a valid
assumption, but it's likely unportable, and *not* obviously correct.
- Replaced empty parameter lists with `(void)` to avoid passing
parameters to it. Empty parameter lists are old style declarations for
unknown number and type of arguments.
- Commented out (as `#if DHT_HARDENING` block) the hardening code that
was never executed.
- Minor style fix: don't use `default` in enum-switches unless the number
of enumerators in the default case is very large. In this case, it was
2, so we want to list them both explicitly to be warned about missing
one if we add one in the future.
- Removed the only two function declarations from nTox.h and put them
into nTox.c. They are not used outside and nTox is not a library.
- Don't cast between object and function pointers.
- Use standard compliant `__VA_ARGS__` in macros.
- Add explicit `__extension__` on unnamed union in struct (it's a GNU
extension).
- Remove ; after function definitions.
- Replace `const T foo = 3;` for integral types `T` with `enum { foo = 3 };`.
Folding integral constants like that as compile time constants is a GNU
extension. Arrays allocated with `foo` as dimension are VLAs on strictly
compliant C99 compilers.
- Replace empty initialiser list `{}` with zero-initialiser-list `{0}`.
The former is a GNU extension meaning the latter.
- Cast `T*` (where `T != void`) to `void *` in format arguments. While any
object pointer can be implicitly converted to and from `void *`, this
conversion does not happen in variadic function calls.
- Replace arithmetic on `void *` with arithmetic on `char *`. The former
is non-compliant.
- Replace non-`int`-derived types (like `uint16_t`, which is
`short`-derived) in bit fields with `int`-derived types. Using any type
other than `int` or `unsigned int` (or any of their aliases) in bit
fields is a GNU extension.
Messenger is slightly twisty when it comes to sending connection status
callbacks It will very likely need at the very least a partial refactor to
clean it up a bit. Toxcore shouldn't need void *userdata as deep as is
currently does.
(amend 1) Because of the nature of toxcore connection callbacks, I decided to
change this commit from statelessness for connections changes to statelessness
for friend requests. It's simpler this was and doesn't include doing anything
foolish in the time between commits.
group fixup because grayhatter doesn't want to do it
"arguably correct" is not how you write security sensitive code
Clear a compiler warning about types within a function.
Also, no longer #include the group code into tox.c. Instead, compile it
separately in tox_group.c. This is a bit less surprising to someone looking
around the code. Having some implementations in a .h file is certainly a bit
surprising to a disciplined C programmer, especially when there is no technical
reason to do it.
This removes the global logger (which by the way was deleted when the first tox
was killed, so other toxes would then stop logging). Various bits of the code
now carry a logger or pass it around. It's a bit less transparent now, but now
there is no need to have a global logger, and clients can decide what to log and
where.
- This PR also adds a DEBUG cmake option that enables -DTOX_DEBUG.
- We also remove `-Wall`, because there are too many warnings, and nobody really
looks at them at the moment. We'll see about fixing them soon. We'll also want
to enable `-Werror` at some point.
- Finally, this PR enables `-O3` to make sure toxcore still works correctly
under heavy compiler optimisations.
If libsodium can't be found with PKG_CHECK_MODULES, try AC_CHECK_LIB. If that
also fails, abort configure. If a user passes --with-libsodium-libs explicitly,
that overrides the pkg-config found location.
**What are we doing?**
We are moving towards stateless callbacks. This means that when registering a
callback, you no longer pass a user data pointer. Instead, you pass a user data
pointer to tox_iterate. This pointer is threaded through the code, passed to
each callback. The callback can modify the data pointed at. An extra indirection
will be needed if the pointer itself can change.
**Why?**
Currently, callbacks are registered with a user data pointer. This means the
library has N pointers for N different callbacks. These pointers need to be
managed by the client code. Managing the lifetime of the pointee can be
difficult. In C++, it takes special effort to ensure that the lifetime of user
data extends at least beyond the lifetime of the Tox instance. For other
languages, the situation is much worse. Java and other garbage collected
languages may move objects in memory, so the pointers are not stable. Tox4j goes
through a lot of effort to make the Java/Scala user experience a pleasant one by
keeping a global array of Tox+userdata on the C++ side, and communicating via
protobufs. A Haskell FFI would have to do similarly complex tricks.
Stateless callbacks ensure that a user data pointer only needs to live during a
single function call. This means that the user code (or language runtime) can
move the data around at will, as long as it sets the new location in the
callback.
**How?**
We are doing this change one callback at a time. After each callback, we ensure
that everything still works as expected. This means the toxcore change will
require 15 Pull Requests.
This allows us to more clearly define interfaces between modules, and have the
linker help us ensure that module boundaries are respected.
The onion/tcp/net_crypto layer is a bit too large. This is due to a cyclic
dependency (onion -> net_crypto -> TCP -> onion). We may or may not want to
break that cycle in the future to allow the onion library to exist on its own
without net_crypto.
Also, fix the hstox build that was taking half an hour. It now takes 5 minutes.
Also, perform distcheck on travis to ensure that make dist works. It's not
actually failing the build at the moment due to broken tests.
This is in preparation for having multiple types of build. One of the future
builds will be a hstox build, another may be frama-c or some other static
analyser. It makes sense to split these up into multiple builds, because each of
them can take a while, and running them in parallel will speed things up. Also,
the hstox test coverage should be reported separately from the toxcore auto_test
coverage.
> increased the timeout for TCP tests because per @irungentoo the network on Travis-CI can be slow sometimes
> allowed groupchats test to restart on error until timeout This had to be done because current groupchats are fundamentally broken and 3/5 times they'll 'net-split' on connect
>> Drop group chat tests, add comment to the reason
> added some debugging information to TCP tests, and a #define to force IPV6 (Travis-CI only uses IPv4 on their containers) and decreased the itr interval
> Went crazy with timeouts for Tox network stuff on Travis. Tests on TCP will still randomly fail due to timeouts. I can't reproduce on any local system. So again per @irungentoo, Travis is slow, let's offer it a short bus.
