See discussion in #46814. Now going with the safe option again (like in 3.2)
as it turns out that we can't rely on user environments on Windows, since each
shell has a different set of env variables (especially the ones necessary to
use MSVC).
SCons does its own magic when we don't pass it an `ENV` dictionary, so we
should preserve it and only add things in a second step.
Fixes this warning when compiling with MSVC using git-bash.exe:
```
Missing environment variable: WindowsSdkDir
```
Possibly fixes build issues when having both MinGW and MSVC installed and an
older SCons version.
This fixes a regression from #46774 where `env["ENV"]` would miss some
important env variables on Windows, such as `SystemRoot`, `PATHEXT`, etc.
To have those, we can either use the default `ENV` created by SCons, or
propagate the whole external environment.
Fixes#46790.
We constructed the SCons environment without taking any (shell) environment
variables into account, and then appended a few, but too late. This would
cause variables like `env[CXX]` not to be properly expanded to respect a
non-standard `PATH`.
With this fix, setting:
```
PATH=$GODOT_SDK/bin:$PATH
```
will now properly use `$GODOT_SDK/bin/gcc` if available over `/usr/bin/gcc`.
The `dev=yes` and `production=yes` options work as aliases to set a number of
options, while still aiming to allow overriding specific options if the user
wishes so. (E.g. `production=yes use_lto=no` should work to enable production
defaults *but* disable LTO.)
That wasn't working as `ARGUMENTS.get()` returns a string and not a boolean as
expected by `BoolVariable`, and this wasn't flagged as a bug... So added a
helper method using SCons' `BoolVariable._text2bool` to do the conversion
manually.
This is meant for users making custom builds to match the options used on
optimized, official builds.
This enables, on the platforms which support them:
- `use_static_cpp=yes` (portable binaries for Linux and Windows)
- `use_lto=yes` (link time optimizations - note: requires a lot of RAM!)
- `debug_symbols=no` (no debug symbols, smaller binaries)
Also abort when using MSVC with `production=yes`, as:
- It cannot optimize the GDScript VM like GCC or Clang do, leading to
significant performance drops.
- Its LTO support is unreliable, at least used to trigger crashes last
we tried it extensively.
All options can still be overridden if specified, and the `dev=yes` option
was changed to also support overrides.
Fixes a pre-existing bug that #44433 exposed.
It's pretty hacky, but we use `platform` in `env` both as an optional command
line option (instead it can be autodetected, or passed via the `p` alias, and
on Linux it might be overridden if you pass one of the convenience alias
values), and as the reference value for what platform we're building on.
Thus we override `env_base["platform"]` with the autodetected or validated
platform, but any call to `opts.Update(env_base)` overrides it with the
original command line option... causing e.g. #44448.
The proper fix would be to refactor all this so that we don't reuse
`env["platform"]` for platform detection (it could instead be e.g.
`env.platform` as a member variable which holds the validated value),
but for now I'm tapering over the immediate breakage.
Fixes#44448 and other breakages induced by #44433.
Otherwise we can get situations where platform-specific opts with the same name
can override each other depending on the order at which platforms are parsed,
as was the case with `use_static_cpp` in Linux/Windows.
Fixes#44304.
This also has the added benefit that the `scons --help` output will now only
include the options which are relevant for the selected (or detected) platform.
ptrcall is now also used to optimize calls in GDScript, on top of the existing
use by the GDNative and Mono modules.
It no longer makes sense to make it optional.
This adds `custom_modules_recursive` which allows to detect and collect
all nested C++ modules which may reside in any directory specified by
`custom_modules` option.
The detection logic is made to be more strict because `SCSub` may be
used for organizing hierarchical builds within a module itself, so the
existence of `register_types.h` and `config.py` is checked as well
(these are all required for a C++ module to be compiled by Godot).
For performance reasons, built-in modules are not checked recursively,
and there's no benefit of doing so in the first place.
It's now possible to specify a directory path pointing to a *single*
module, as it may contain nested modules which are detected recursively.
This code currently isn't compiled (and cannot compile).
We plan to re-add OpenGL ES-based renderer(s) in Godot 4.0 alongside Vulkan
(probably ES 3.0, possibly also a low-end ES 2.0), but the code will be quite
different so it's not relevant to keep this old Godot 3.2 code.
The `drivers/gles2` code from the `3.2` branch can be used as a reference for
a potential new implementation.
A new `env.Run` method is added which allows to control the verbosity
of builders output automatically depending on whether the "verbose"
option is set. It also allows to optionally run any SCons commands in a
subprocess using the existing `run_in_subprocess` method, unifying
the interface. `Action` objects wrap all builder functions to include a
short build message associated with any action.
Notably, this removes quite verbose output generated by `make_doc_header`
and `make_editor_icons_action` builders.
Until https://github.com/psf/black/pull/1328 makes it in a stable release,
we have to use the latest from Git.
Apply new style fixes done by latest black.
Allows switching `tests=yes`/`no` and rebuilding only tests and main,
instead of the whole engine.
Co-authored-by: Andrii Doroshenko (Xrayez) <xrayez@gmail.com>
Implements exit codes into the engine so tests can return their statuses.
Ideally we don't do this, and we use FIXUP logic to 'begin' and 'end' the engine execution for tests specifically.
Since realistically we're initialising the engine here we don't want to do that, since String should not require an engine startup to test a single header.
This lowers the complexity of running the unit tests and even for
physics should be possible to implement such a fix.
There are too many users who compile Godot from source and are not familiar
with the buildsystem or C/C++ compilation warnings, and thus report any kind
of yet-unfixed warning as a (often duplicate) bug.
Compiler warnings change at every compiler version and are different for each
compiler, so it's difficult to ensure that the codebase would always be 100%
warning-free, especially in the future.
I already disabled it for stable releases in #37958, but having it on non
stable commits could also become an annoyance in the future when trying to
bisect issues with a new compiler version which emits warnings unknown at
the time of commit.
TL;DR: Contributors, use `dev=yes` or `werror=yes`. CI does and won't let you
create new warnings ;)
A new `methods.dump(env)` is added to dump the construction environment
used by SCons to build Godot to a `.scons_env.json`. The file can be used
for debugging purposes and any external tool.
This is still a bit hacky and eventually we should rework the way we handle
optional dependencies (especially with regard to builtin/system libs), but
it's a simple first step.
Fixes#39219.
The `custom_modules` option was only read via the command line
by fetching `ARGUMENTS` dictionary directly.
Instead, the option's value can now be read via any existing
configuration files (`custom.py`) as well as command line, while also
updating the environment.
Sconscript provides it's own `Exit()` method which is currently
an alias for `sys.exit()` internally, with the only difference that if
no exit code is specified, it defaults to 0.
This encourages the usage of SCons-implemented methods like
`Glob()` over `glob.glob()`, which may overcome limitations of the
built-in Python features in the future.
The insertion order for dictionaries is only a language feature for
Python 3.6/3.7+ implementations, and not prior to that.
This ensures that the engine won't be rebuilt if the order of detected
modules changes in any way, as the `OrderedDict` should guarantee
inerstion order.
This patch adds ability to include external, user-defined C++ modules
to be compiled as part of Godot via `custom_modules` build option
which can be passed to `scons`.
```
scons platform=x11 tools=yes custom_modules="../project/modules"
```
Features:
- detects all available modules under `custom_modules` directory the
same way as it does for built-in modules (not recursive);
- works with both relative and absolute paths on the filesystem;
- multiple search paths can be specified as a comma-separated list.
Module custom documentation and editor icons collection and generation
process is adapted to work with absolute paths needed by such modules.
Also fixed doctool bug mixing absolute and relative paths respectively.
Implementation details:
- `env.module_list` is a dictionary now, which holds both module name as
key and either a relative or absolute path to a module as a value.
- `methods.detect_modules` is run twice: once for built-in modules, and
second for external modules, all combined later.
- `methods.detect_modules` was not doing what it says on the tin. It is
split into `detect_modules` which collects a list of available modules
and `write_modules` which generates `register_types` sources for each.
- whether a module is built-in or external is distinguished by relative
or absolute paths respectively. `custom_modules` scons converter
ensures that the path is absolute even if relative path is supplied,
including expanding user paths and symbolic links.
- treats the parent directory as if it was Godot's base directory, so
that there's no need to change include paths in cases where custom
modules are included as dependencies in other modules.
There's a builtin `toolpath` option we can use for that, so no need to hack
around a custom `scons_site` path.
The script requires SCons 3.1.1 or later, so we enable it conditionally.
Follow-up to #32848.
This tool is originally from mongodb.
- Updated CPPSUFFIXES to use scons suffixes
- objective-c files will also be loaded into the compilation database where the compiler / tooling is available to compile the files.
Known limitations:
- This will not work with msvc as your compiler.
-Added LocalVector (needed it)
-Added stb_rect_pack (It's pretty cool, we could probably use it for other stuff too)
-Fixes and changes all around the place
-Added library for 128 bits fixed point (required for Delaunay3D)
Stable releases are tagged and need to stay easy to compile in the future.
As new compiler versions introduce new warnings or catch more occurrences,
have -Werror set in tagged releases could be a bother.
We still want it on by default for all Godot developers, so it's now
conditional.
After an effort spanning several years, we should now be warning-free
on all major compilers, so we can set `-Werror` to ensure that we don't
introduce warnings in new code.
Disable -Werror=strict-overflow on GCC 7 though, as it seems bogus and
was fixed in 8+.
Some required changes are made:
- locally imported SCons-specific packages within the method;
- `global` variables converted to `nonlocal` (used in nested functions).
Configured for a max line length of 120 characters.
psf/black is very opinionated and purposely doesn't leave much room for
configuration. The output is mostly OK so that should be fine for us,
but some things worth noting:
- Manually wrapped strings will be reflowed, so by using a line length
of 120 for the sake of preserving readability for our long command
calls, it also means that some manually wrapped strings are back on
the same line and should be manually merged again.
- Code generators using string concatenation extensively look awful,
since black puts each operand on a single line. We need to refactor
these generators to use more pythonic string formatting, for which
many options are available (`%`, `format` or f-strings).
- CI checks and a pre-commit hook will be added to ensure that future
buildsystem changes are well-formatted.
- Improve the SCsub to allow unbundling and remove unnecessary code.
- Move files around to match upstream source.
- Re-sync with upstream commit 308db73d0b3c2d1870cd3e465eaa283692a4cf23
to ensure we don't have local modifications.
- Doesn't actually build against current version 5.0.1 due to the lack
of the new ArmaturePopulate API that Gordon authored. We'll have to
wait for a public release with that API (5.1?) to enable unbundling.
This attribute is now part of the standard we target so we no longer
need compiler-specific hacks.
Also enables -Wimplicit-fallthrough for Clang now that we can properly
support it. It's already on by default for GCC's -Wextra.
Fixes new warnings raised by Clang's -Wimplicit-fallthrough.
As per #36436, we now need C++17's guaranteed copy elision feature to
solve ambiguities in Variant.
Core developers discussed the idea to move from C++14 to C++17 as our
minimum required C++ standard, and all agreed. Note that this doesn't
mean that Godot is going to be written in "modern C++", but we'll use
modern features where they make sense to simplify our "C with classes"
codebase. Apart from new code written recently, most of the codebase
still has to be ported to use newer features where relevant.
Proper support for C++17 means that we need recent compiler versions:
- GCC 7+
- Clang 6+
- VS 2017 15.7+
Additionally, C++17's `std::shared_mutex` (conditionally used by
`vk_mem_alloc.h` when C++17 support is enabled) is only available in
macOS 10.12+, so we increase our minimum supported version.
On GCC and Clang, we use C11 and C++14 with GNU extensions (`std=gnu11`
and `std=gnu++14`). Those are the defaults for current GCC and Clang,
and also match the feature sets we want to use in Godot.
On MSVC, we require C++14 support explicitly with `/std:c++14`, and
make it strict with the use of `/permissive-` (so features of C++17 or
later can't be used).
Moves the definition before querying environment flags and platform
config so that it can be overridden when necessary.
Fix -Wunused-variable, -Wunused-but-set-variable and -Wswitch warnings
raised by GCC 8 and 9.
Fix -Wunused-function, -Wunused-private-field and
-Wtautological-constant-out-of-range-compare raised by Clang.
Fix MSVC 2019 warning C4804 (unsafe use of type 'bool' in comparison
operation).
GCC -Wcpp warnings/Clang -W#warnings (`#warning`) are no longer raising
errors and will thus not abort compilation with `werror=yes`.
Treat glslang headers are system headers to avoid raising warnings.
Re-enables us to build with `werror=yes` on Linux and macOS, thus
catching warnings that would be introduced by new code.
Fixes#36132.
Due to the port to Vulkan and complete redesign of the rendering backend,
the `drivers/gles3` code is no longer usable in this state and is not
planned to be ported to the new architecture.
The GLES2 backend is kept (while still disabled and non-working) as it
will eventually be ported to serve as the low-end renderer for Godot 4.0.
Some GLES3 features might be selectively ported to the updated GLES2
backend if there's a need for them, and extensions we can use for that.
So long, OpenGL driver bugs!
- Renamed option to `builtin_vulkan`, since that's the name of the
library and if we were to add new components, we'd likely use that
same option.
- Merge `vulkan_loader/SCsub` in `vulkan/SCsub`.
- Accordingly, don't use built-in Vulkan headers when not building
against the built-in loader library.
- Drop Vulkan registry which we don't appear to need currently.
- Style and permission fixes.
We already had `MODULE_*_ENABLED` defines but only in the modules
environment, and a few custom `*_ENABLED` defines in the main env
when we needed the information in core.
Now this is defined in a single header which can be included in the
files that need this information.
The new 'split_libmodules=yes' option is useful to work around linker
command line size limitations when linking a huge number of objects.
We're currently over 64k chars when linking libmodules.a on Windows
with MinGW, which triggers issues as seen in #30892.
Even on Linux, we can also reach linker command line size limitations
by adding more custom modules.
We force this option to True for MinGW on Windows, which fixes#30892.
Additional changes to lib splitting:
- Fix linking of the split module libs with interdependent symbols,
hacking our way into LINKCOM and SHLINKCOM to set the `--start-group`
and `--end-group` flags.
- Fix Python 3 compatibility in `methods.split_lib()`.
- Drop seemingly obsolete condition for 'msys' on 'posix'.
- Drop the unnecessary 'split_drivers' as the drivers lib is no longer
too big since we moved all thirdparty builds to modules.
Co-authored-by: Hein-Pieter van Braam-Stewart <hp@tmm.cx>
Third-party platforms (e.g. console ports) need to be able to
disable JIT support in the regex module too, so it can't be
hardcoded in the module SCsub. This is cleaner this way anyway.
Fixes#19316.
- Add or remove the necessary subdirectorires to the includes to remove
dependency on the editor directory being in the build's include path.
- Ensure includes in modified files conform to style guideline.
- Remove editor from the build include path.
**Important:** This does not mean *yet* that C++11 features should be used
in contributions to Godot's codebase.
For now this change is done solely for feature branches working on Vulkan
support and GDScript typed instruction sets for Godot 4.0, which will both
use C++11 features and are based on the master branch.
The plan is to start porting the codebase to C++11 after Godot 3.2 is
released, following upcoming guidelines on the subset of new features that
should be used, and when/how to use them.
We will advertise clearly when C++11 contributions are open, especially
once we start a coordinated effort to port Godot's massive codebase. In the
meantime, please bear with us and good ol' C++03. :)
The functionality is similar to how `doc_classes` are retrieved per module.
The build system will search for custom icons path defined per module via
`get_icons_path()` method in `config.py` or default icons path.
If such paths don't exist, only the editor's own icons will be built.
Most module icons were moved from editor/icons to respective modules.
This can help to solve compilation issues on systems with Japanese
locale with encodings like Shift_JIS and UTF-8-BOM.
Also be more consistent using String::utf8() to represent cyrilic
unicode characters in test_string.cpp
Clarified some comments in test_string.cpp for some Unicode characters.
We're close to being able to compile all platforms with -Werror -Wextra,
so it's best if developers use those options when compiling their code
as those checks are also done on CI.
Also enabled -Wduplicated-branches on GCC.
Include paths are processed from left to right, so we use Prepend to
ensure that paths to bundled thirdparty files will have precedence over
system paths (e.g. `/usr/include` should have lowest priority).
Many contributors (me included) did not fully understand what CCFLAGS,
CXXFLAGS and CPPFLAGS refer to exactly, and were thus not using them
in the way they are intended to be.
As per the SCons manual: https://www.scons.org/doc/HTML/scons-user/apa.html
- CCFLAGS: General options that are passed to the C and C++ compilers.
- CFLAGS: General options that are passed to the C compiler (C only;
not C++).
- CXXFLAGS: General options that are passed to the C++ compiler. By
default, this includes the value of $CCFLAGS, so that setting
$CCFLAGS affects both C and C++ compilation.
- CPPFLAGS: User-specified C preprocessor options. These will be
included in any command that uses the C preprocessor, including not
just compilation of C and C++ source files [...], but also [...]
Fortran [...] and [...] assembly language source file[s].
TL;DR: Compiler options go to CCFLAGS, unless they must be restricted
to either C (CFLAGS) or C++ (CXXFLAGS). Preprocessor defines go to
CPPFLAGS.
