Commit graph

135 commits

Author SHA1 Message Date
Hristo Stamenov
d6ada73063
Fix import for collections Mapping
Mapping and other "abstract base classes" were moved after python 3.3 from collections to collections.abc
Python 3.3 is long gone and a newer version of python won't support this code.

(cherry picked from commit 7ec6392d46)
2021-11-15 16:27:19 +01:00
Edward Auttonberry
4c76594e0d
Variants and targets are generated instead of manually defined. The specific details for each configuration are logically separated from the generators
Add support for specifying defines for each configuration

Add support for specifying extra cli args for each configuration

Add support for specifying extra includes for each configuration

(cherry picked from commit 4c2f0a5c94)
2021-11-15 16:24:34 +01:00
Rémi Verschelde
a5093d64ac
SCons: List .gen.cpp sources explicitly to avoid globbing errors
Whenever we change the name (or remove) generated cpp files with the `.gen.cpp`
extension, users run into build issues when switching between branches (i.e.
switching before and after the name change/removal). This is because we glob
`*.cpp` so if a now-obsolete file from a previous build is present, we'll
include it too, potentially leading to bugs or compilation failure (due to
missing headers or invalid code).

So globbing patterns in `add_source_files` will now skip files ending with
`.gen.cpp`, which should instead be passed explicitly where they're used.

(cherry picked from commit c133480531)
2021-10-16 10:59:14 +02:00
Rémi Verschelde
8605d4ec26
SCons: Fix potential error when pruning cache on CI
This could cause spurious errors on CI when trying to prune the cache,
as for some reason it tries to remove files/paths which do not exist.

That points at a bug in the `cache_progress` logic but at least this
workaround should prevent CI failures.

(cherry picked from commit 825b245f0d)
2021-08-18 23:05:58 +02:00
Rémi Verschelde
e7f7d5f330
SCons: Add method to detect Emscripten and use it for warnings config
Emscripten is LLVM-based so we want to follow the same logic. But we can't just
put it as a match in `methods.using_clang()` as that would mess with the
compiler version detection logic used to restrict old GCC and Clang releases.

(cherry picked from commit 34421683eb)
2021-08-12 16:37:38 +02:00
Rémi Verschelde
91dfb803c5
SCons: Fix info message when defining GODOT_VERSION_STATUS
(cherry picked from commit e92ba89dc7)
2021-08-05 16:36:20 +02:00
Rémi Verschelde
a43365bd58
Allow overriding VERSION_STATUS with GODOT_VERSION_STATUS in env
`VERSION_STATUS` is part of what constitutes the reference version for a given
Godot build, and is part of the version check for compatible export templates.

For dev snapshots (alpha, beta, RCs), we usually set the `VERSION_STATUS` to
a specific build number (e.g. `beta2`), but this change doesn't end up
committed to the Git repository as we don't want to keep changing `version.py`
for testing builds.

So this new environment override will be what can be used in official builds
and by users making custom builds for specific snapshots.

(cherry picked from commit 948dcb63ca)
2021-08-03 10:20:19 +02:00
Rémi Verschelde
cbbea6084d
SCons: Generate header with info on which modules are enabled
We already had `MODULE_*_ENABLED` defines but only in the modules
environment, and a few custom `*_ENABLED` defines in the main env
when we needed the information in core.

Now this is defined in a single header which can be included in the
files that need this information.

(cherry picked from commit b7297fb39c)
2021-07-14 23:09:47 +02:00
Rémi Verschelde
92d59d0879 SCons: Properly handle overriding default values to bool options
The `dev=yes` and `production=yes` options work as aliases to set a number of
options, while still aiming to allow overriding specific options if the user
wishes so. (E.g. `production=yes use_lto=no` should work to enable production
defaults *but* disable LTO.)

That wasn't working as `ARGUMENTS.get()` returns a string and not a boolean as
expected by `BoolVariable`, and this wasn't flagged as a bug... So added a
helper method using SCons' `BoolVariable._text2bool` to do the conversion
manually.
2021-02-24 10:20:46 +01:00
Rémi Verschelde
10f0be7719
SCons: Fix Godot detection in custom modules logic
`exec()` was not a good idea as it assumes a certain type of `version.py` file
similar to Godot's own file, which is not always a reliable assumption (see
https://github.com/godotengine/godot/pull/43057#issuecomment-777632900).

Also restores Python 2 support for the 3.2 branch.

(cherry picked from commit 75910d1e9b)
2021-02-12 11:09:40 +01:00
Andrii Doroshenko (Xrayez)
65a2f0dfd4
SCons: Add an option to detect C++ modules recursively
This adds `custom_modules_recursive` which allows to detect and collect
all nested C++ modules which may reside in any directory specified by
`custom_modules` option.

The detection logic is made to be more strict because `SCSub` may be
used for organizing hierarchical builds within a module itself, so the
existence of `register_types.h` and `config.py` is checked as well
(these are all required for a C++ module to be compiled by Godot).

For performance reasons, built-in modules are not checked recursively,
and there's no benefit of doing so in the first place.

It's now possible to specify a directory path pointing to a *single*
module, as it may contain nested modules which are detected recursively.

(cherry picked from commit a3c2c1e18a)
2021-02-08 22:36:37 +01:00
Marcel Admiraal
e21adf2bc6 Don't handle BaseException in build scripts 2020-12-12 10:10:23 +00:00
bruvzg
b338ba1b27
[3.2] Do not raise exception if iOS SDK is not installed. 2020-10-26 14:15:44 +02:00
bruvzg
d44bc8d5bc
[3.2] Add iOS SDK version check. 2020-10-06 11:33:52 +03:00
Bartłomiej T. Listwon
858bdbacfd
Add all headers to VS Project
(cherry picked from commit 317c2b194d)
2020-09-29 13:57:56 +02:00
Bartłomiej T. Listwon
405412938a Fix custom modules in Visual Studio
Without `custom_modules` scons command in VS project was broken
(fixes regression from commit 6122a504ee)
2020-09-18 18:04:37 +02:00
Rémi Verschelde
217dbdc9e7
SCons: Fix regression in Python 2 support
Introduced in 7e1312055b by not using
the compat method.

Fixes #41858.
2020-09-07 20:29:21 +02:00
Andrii Doroshenko (Xrayez)
6122a504ee Build the engine with custom modules in Visual Studio
First, compile the engine normally with:
```
scons custom_modules="path/to/your/modules" vsproj=yes
```
Then run the Visual Studio project. You can now rebuild the engine if
you need to make changes to custom modules directly within IDE.

(cherry picked from commit 6eb4b89eef)
2020-09-04 08:46:54 +02:00
Tomasz Chabora
7e1312055b Fix emoji branch compilation error
(cherry picked from commit 2407562366)
2020-07-24 10:31:57 +02:00
Andrii Doroshenko (Xrayez)
670205a22f Expand env vars for custom_modules build option
The order of conversion is also changed to ensure that the resulting
path can be properly validated later on.

(cherry picked from commit 80249e7190)
2020-07-13 15:13:14 +02:00
Rémi Verschelde
c87b4f9d71 SCons: Fix python2 compatibility after #37198 and #37248 2020-06-11 20:15:48 +02:00
Andrii Doroshenko (Xrayez)
23ef1e0f70 SCons: Dump construction environment to a file
A new `methods.dump(env)` is added to dump the construction environment
used by SCons to build Godot to a `.scons_env.json`. The file can be used
for debugging purposes and any external tool.

(cherry picked from commit 42bee75e86)
2020-06-10 15:30:52 +02:00
Andrii Doroshenko (Xrayez)
bc21cd8592 Moved to methods.py as a show_progress method.
Some required changes are made:
- locally imported SCons-specific packages within the method;
- `global` variables converted to `nonlocal` (used in nested functions).

(cherry picked from commit d753a7630a)
2020-06-10 15:30:52 +02:00
Rémi Verschelde
7bf9787921 SCons: Format buildsystem files with psf/black
Configured for a max line length of 120 characters.

psf/black is very opinionated and purposely doesn't leave much room for
configuration. The output is mostly OK so that should be fine for us,
but some things worth noting:

- Manually wrapped strings will be reflowed, so by using a line length
  of 120 for the sake of preserving readability for our long command
  calls, it also means that some manually wrapped strings are back on
  the same line and should be manually merged again.

- Code generators using string concatenation extensively look awful,
  since black puts each operand on a single line. We need to refactor
  these generators to use more pythonic string formatting, for which
  many options are available (`%`, `format` or f-strings).

- CI checks and a pre-commit hook will be added to ensure that future
  buildsystem changes are well-formatted.

(cherry picked from commit cd4e46ee65)
2020-06-10 15:30:52 +02:00
Andrii Doroshenko (Xrayez)
d201163557 Do not rely on the existence of config.py while detecting modules
The existence of `SCsub` is checked instead. This file is required for
all modules, and prevents the build system to leave modules without
`config.py` undetected, leading to silently ignoring the module during
compilation.

(cherry picked from commit 12389b3736)
2020-06-04 12:09:39 +02:00
Andrii Doroshenko (Xrayez)
0138ba59ac SCons: use OrderedDict to ensure insertion order of modules
The insertion order for dictionaries is only a language feature for
Python 3.6/3.7+ implementations, and not prior to that.

This ensures that the engine won't be rebuilt if the order of detected
modules changes in any way, as the `OrderedDict` should guarantee
inerstion order.
2020-05-28 17:13:50 +03:00
Andrii Doroshenko (Xrayez)
133997654c Add custom_modules build option to compile external user modules
This patch adds ability to include external, user-defined C++ modules
to be compiled as part of Godot via `custom_modules` build option
which can be passed to `scons`.

```
scons platform=x11 tools=yes custom_modules="../project/modules"
```

Features:

- detects all available modules under `custom_modules` directory the
same way as it does for built-in modules (not recursive);
- works with both relative and absolute paths on the filesystem;
- multiple search paths can be specified as a comma-separated list.

Module custom documentation and editor icons collection and generation
process is adapted to work with absolute paths needed by such modules.
Also fixed doctool bug mixing absolute and relative paths respectively.

Implementation details:

- `env.module_list` is a dictionary now, which holds both module name as
  key and either a relative or absolute path to a module as a value.
- `methods.detect_modules` is run twice: once for built-in modules, and
  second for external modules, all combined later.
- `methods.detect_modules` was not doing what it says on the tin. It is
  split into `detect_modules` which collects a list of available modules
  and `write_modules` which generates `register_types` sources for each.
- whether a module is built-in or external is distinguished by relative
  or absolute paths respectively. `custom_modules` scons converter
  ensures that the path is absolute even if relative path is supplied,
  including expanding user paths and symbolic links.
- treats the parent directory as if it was Godot's base directory, so
  that there's no need to change include paths in cases where custom
  modules are included as dependencies in other modules.

(cherry picked from commit a96f0e98d7)
2020-05-25 17:08:38 +03:00
unknown
6e6db2d9cb Remove the dead function win32_spawn from methods.py.
(cherry picked from commit d78a78bc81)
2020-03-25 11:38:54 +01:00
Rémi Verschelde
75164169c4 SCons: Fix get_compiler_version() to return ints
Otherwise comparisons would fail for compiler versions above 10.
Also simplified code somewhat to avoid using subprocess too much
needlessly.

(cherry picked from commits c7dc5142b5
and df7ecfc4a7)
2020-03-04 14:40:12 +01:00
Joost Heitbrink
dc61323b2c PCK: Set VERSION_PATCH in header, factor out header magic
Unify pack file version and magic to avoid hardcoded literals.

`version.py` now always includes `patch` even for the first release in
a new stable branch (e.g. 3.2). The public name stays without the patch
number, but `Engine.get_version_info()` already included `patch == 0`,
and we can remove some extra handling of undefined `VERSION_PATCH` this
way.

Co-authored-by: Rémi Verschelde <rverschelde@gmail.com>
2020-01-06 13:13:17 +01:00
unknown
0ab7bd402a Remove unused imports 2020-01-03 01:02:38 +05:30
Rémi Verschelde
c320a82213 SCons: Add 'split_libmodules' option to workaround linker issue
The new 'split_libmodules=yes' option is useful to work around linker
command line size limitations when linking a huge number of objects.
We're currently over 64k chars when linking libmodules.a on Windows
with MinGW, which triggers issues as seen in #30892.

Even on Linux, we can also reach linker command line size limitations
by adding more custom modules.

We force this option to True for MinGW on Windows, which fixes #30892.

Additional changes to lib splitting:

- Fix linking of the split module libs with interdependent symbols,
  hacking our way into LINKCOM and SHLINKCOM to set the `--start-group`
  and `--end-group` flags.
- Fix Python 3 compatibility in `methods.split_lib()`.
- Drop seemingly obsolete condition for 'msys' on 'posix'.
- Drop the unnecessary 'split_drivers' as the drivers lib is no longer
  too big since we moved all thirdparty builds to modules.

Co-authored-by: Hein-Pieter van Braam-Stewart <hp@tmm.cx>
2019-12-11 15:40:28 +01:00
santouits
21297a533e fix getting correct mingw-w64 version
When checking mingw-w64 version, at least on debian, the regex being used returned 86 because the name of the binary in debian starts with x86_64-w64 so we use the dumpversion option that gcc has. This fixes not compiling because gcc versions < 7 don't have some checks like shadow-local
2019-07-26 00:04:16 +03:00
Rémi Verschelde
66d09a6b4c SCons: Fix uses of [].append instead of env.add_source_files()
Also added support for SCons project-absolute paths (starting with #) and
warning about duplicates in add_source_files(), and fixed
default_controller_mappings.gen.cpp being included twice after first build
due to *.cpp globbing.

Part of #30270.
2019-07-22 15:08:32 +02:00
Rémi Verschelde
b0d41847ed SCons: Use CPPDEFINES instead of CPPFLAGS for pre-processor defines
It's the recommended way to set those, and is more portable
(automatically prepends -D for GCC/Clang and /D for MSVC).

We still use CPPFLAGS for some pre-processor flags which are not
defines.
2019-07-03 09:59:04 +02:00
hbina085
f4e1ed7e63 some variables in methods.py are not used
Thus they can be safely ignored
2019-06-29 06:56:57 -04:00
Rémi Verschelde
b10dd110e5 Print engine version to stdout when starting Godot
Also include website URL and make it configurable via version.py
together with the rest of the engine branding.

Add mention to MIT license in --help output.
2019-05-28 11:19:21 +02:00
Rémi Verschelde
c2a669a9f0 SCons: Review uses of CCFLAGS, CXXFLAGS and CPPFLAGS
Many contributors (me included) did not fully understand what CCFLAGS,
CXXFLAGS and CPPFLAGS refer to exactly, and were thus not using them
in the way they are intended to be.

As per the SCons manual: https://www.scons.org/doc/HTML/scons-user/apa.html

- CCFLAGS: General options that are passed to the C and C++ compilers.
- CFLAGS: General options that are passed to the C compiler (C only;
  not C++).
- CXXFLAGS: General options that are passed to the C++ compiler. By
  default, this includes the value of $CCFLAGS, so that setting
  $CCFLAGS affects both C and C++ compilation.
- CPPFLAGS: User-specified C preprocessor options. These will be
  included in any command that uses the C preprocessor, including not
  just compilation of C and C++ source files [...], but also [...]
  Fortran [...] and [...] assembly language source file[s].

TL;DR: Compiler options go to CCFLAGS, unless they must be restricted
to either C (CFLAGS) or C++ (CXXFLAGS). Preprocessor defines go to
CPPFLAGS.
2019-04-24 16:57:58 +02:00
Rémi Verschelde
8d867cf7c5 Also disable C and C++ specific warnings in thirdparty code
Move the `Append` up to make sure that the keys exist and avoid the
need to check `if CPPFLAGS in self`, etc.
2019-04-24 07:41:00 +02:00
Juan Linietsky
dd03dcbd5a Android now (optionally) builds the template when exporting
Added new way to create add-ons
Removed old way to create add-ons
2019-04-07 15:46:52 -03:00
Hendrikto
49a81308c0 Remove unused imports 2019-04-06 18:05:05 +02:00
Rémi Verschelde
e4a96164b6 SCons: add methods.using_clang to check used compiler
Also rename `use_gcc` to `using_gcc` to make it clear that it returns
a config but does not alter it.
2019-04-05 12:51:15 +02:00
Rémi Verschelde
195c5e0925 Move YEAR definition to version.py
If it needs to be hardcoded (for the sake of reproducible builds),
it should be together with the other hardcoded version info.

And yeah, two months in, let's move to 2019.
2019-03-05 23:07:03 +01:00
marxin
0d2a105e6b Come up with use_gcc.
Add new method. Fix wrong version condition for -fpie.
2019-02-23 09:17:11 +01:00
marxin
8d51618949 Add -Wshadow=local to warnings and fix reported issues.
Fixes #25316.
2019-02-20 19:44:12 +01:00
George Marques
874cbefa26 Get Git commit hash when Godot is a submodule
Submodules don't have a .git folder in the same place, but a .git file
that points to the actual folder. This change take this into account.
2019-02-08 16:17:07 -02:00
Naoto Kondo
577e547b6a [macOS] Fixed a problem sdk path could not be detected 2018-11-26 19:42:17 +09:00
Rémi Verschelde
a2a5793e13
Merge pull request #21339 from Placinta/master
Fix regular macOS build by passing -isysroot to compiler so correct system headers are found
2018-11-20 14:11:13 +01:00
Rémi Verschelde
173b342ca7 Remove trailing whitespace
With `sed -i $(rg -l '[[:blank:]]*$' -g'!thirdparty') -e 's/[[:blank:]]*$//g'`
(+ manual revert of some thirdparty code under `platform/android`).
2018-11-20 11:15:02 +01:00
lupoDharkael
edcca5f7ad Dont use equality operators with None singleton in python files 2018-10-27 01:18:15 +02:00