This patch adds ability to include external, user-defined C++ modules
to be compiled as part of Godot via `custom_modules` build option
which can be passed to `scons`.
```
scons platform=x11 tools=yes custom_modules="../project/modules"
```
Features:
- detects all available modules under `custom_modules` directory the
same way as it does for built-in modules (not recursive);
- works with both relative and absolute paths on the filesystem;
- multiple search paths can be specified as a comma-separated list.
Module custom documentation and editor icons collection and generation
process is adapted to work with absolute paths needed by such modules.
Also fixed doctool bug mixing absolute and relative paths respectively.
Implementation details:
- `env.module_list` is a dictionary now, which holds both module name as
key and either a relative or absolute path to a module as a value.
- `methods.detect_modules` is run twice: once for built-in modules, and
second for external modules, all combined later.
- `methods.detect_modules` was not doing what it says on the tin. It is
split into `detect_modules` which collects a list of available modules
and `write_modules` which generates `register_types` sources for each.
- whether a module is built-in or external is distinguished by relative
or absolute paths respectively. `custom_modules` scons converter
ensures that the path is absolute even if relative path is supplied,
including expanding user paths and symbolic links.
- treats the parent directory as if it was Godot's base directory, so
that there's no need to change include paths in cases where custom
modules are included as dependencies in other modules.
Configured for a max line length of 120 characters.
psf/black is very opinionated and purposely doesn't leave much room for
configuration. The output is mostly OK so that should be fine for us,
but some things worth noting:
- Manually wrapped strings will be reflowed, so by using a line length
of 120 for the sake of preserving readability for our long command
calls, it also means that some manually wrapped strings are back on
the same line and should be manually merged again.
- Code generators using string concatenation extensively look awful,
since black puts each operand on a single line. We need to refactor
these generators to use more pythonic string formatting, for which
many options are available (`%`, `format` or f-strings).
- CI checks and a pre-commit hook will be added to ensure that future
buildsystem changes are well-formatted.
The functionality is similar to how `doc_classes` are retrieved per module.
The build system will search for custom icons path defined per module via
`get_icons_path()` method in `config.py` or default icons path.
If such paths don't exist, only the editor's own icons will be built.
Most module icons were moved from editor/icons to respective modules.
- Refactored all builder (make_*) functions into separate Python modules along to the build tree
- Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere
- Introduced stub to use the builders module as a stand alone script and invoke a selected function
There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp)
on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky
builds. Running all such content generators in a new subprocess instead of directly inside the
build script works around the issue.
Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle.
Suggested workaround did not fully work either.
Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of
running a cross-compilation on Windows they would still be used, but likely it will not happen
in practice. What counts is that the build itself is running on which platform, not the target
platform.
Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.
- The Windows, UWP, Android (on Windows) and Linux builds are
tested with Scons 3.0 alpha using Python 3.
- OSX and iOS should hopefully work but are not tested since
I don't have a Mac.
- Builds using SCons 2.5 and Python 2 should not be impacted.
The other subfolders of tools/ had already been moved to either
editor/, misc/ or thirdparty/, so the hiding the editor code that
deep was no longer meaningful.
