Finalizers are longer guaranteed to be called on exit now that
we switched to .NET Core. This results in native instances leaking.
The only solution I can think of so far is to keep a list of all
instances alive to dispose when the AssemblyLoadContext.Unloading
event is raised.
We're targeting .NET 5 for now to make development easier while
.NET 6 is not yet released.
TEMPORARY REGRESSIONS
---------------------
Assembly unloading is not implemented yet. As such, many Godot
resources are leaked at exit. This will be re-implemented later
together with assembly hot-reloading.
The main focus here was to remove the majority of code that relied on
Mono's embedding APIs, specially the reflection APIs. The embedding
APIs we still use are the bare minimum we need for things to work.
A lot of code was moved to C#. We no longer deal with any managed
objects (`MonoObject*`, and such) in native code, and all marshaling
is done in C#.
The reason for restructuring the code and move away from embedding APIs
is that once we move to .NET Core, we will be limited by the much more
minimal .NET hosting.
PERFORMANCE REGRESSIONS
-----------------------
Some parts of the code were written with little to no concern about
performance. This includes code that calls into script methods and
accesses script fields, properties and events.
The reason for this is that all of that will be moved to source
generators, so any work prior to that would be a waste of time.
DISABLED FEATURES
-----------------
Some code was removed as it no longer makes sense (or won't make sense
in the future).
Other parts were commented out with `#if 0`s and TODO warnings because
it doesn't make much sense to work on them yet as those parts will
change heavily when we switch to .NET Core but also when we start
introducing source generators.
As such, the following features were disabled temporarily:
- Assembly-reloading (will be done with ALCs in .NET Core).
- Properties/fields exports and script method listing (will be
handled by source generators in the future).
- Exception logging in the editor and stack info for errors.
- Exporting games.
- Building of C# projects. We no longer copy the Godot API assemblies
to the project directory, so MSBuild won't be able to find them. The
idea is to turn them into NuGet packages in the future, which could
also be obtained from local NuGet sources during development.
We will be progressively moving most code to C#.
The plan is to only use Mono's embedding APIs to set things at launch.
This will make it much easier to later support CoreCLR too which
doesn't have rich embedding APIs.
Additionally the code in C# is more maintainable and makes it easier
to implement new features, e.g.: runtime codegen which we could use to
avoid using reflection for marshaling everytime a field, property or
method is accessed.
SOME NOTES ON INTEROP
We make the same assumptions as GDNative about the size of the Godot
structures we use. We take it a bit further by also assuming the layout
of fields in some cases, which is riskier but let's us squeeze out some
performance by avoiding unnecessary managed to native calls.
Code that deals with native structs is less safe than before as there's
no RAII and copy constructors in C#. It's like using the GDNative C API
directly. One has to take special care to free values they own.
Perhaps we could use roslyn analyzers to check this, but I don't know
any that uses attributes to determine what's owned or borrowed.
As to why we maily use pointers for native structs instead of ref/out:
- AFAIK (and confirmed with a benchmark) ref/out are pinned
during P/Invoke calls and that has a cost.
- Native struct fields can't be ref/out in the first place.
- A `using` local can't be passed as ref/out, only `in`. Calling a
method or property on an `in` value makes a silent copy, so we want
to avoid `in`.
REGARDING THE BUILD SYSTEM
There's no longer a `mono_glue=yes/no` SCons options. We no longer
need to build with `mono_glue=no`, generate the glue and then build
again with `mono_glue=yes`. We build only once and generate the glue
(which is in C# now).
However, SCons no longer builds the C# projects for us. Instead one
must run `build_assemblies.py`, e.g.:
```sh
%godot_src_root%/modules/mono/build_scripts/build_assemblies.py \
--godot-output-dir=%godot_src_root%/bin \
--godot-target=release_debug`
```
We could turn this into a custom build target, but I don't know how
to do that with SCons (it's possible with Meson).
OTHER NOTES
Most of the moved code doesn't follow the C# naming convention and
still has the word Mono in the names despite no longer dealing with
Mono's embedding APIs. This is just temporary while transitioning,
to make it easier to understand what was moved where.
The function tried to rearrange properties but that lead to problems with duplication or deleted properties. Implemented the logic that that function did inside the get_property_list both for tool scripts and non-tool scripts.
- RPC configurations are now dictionaries.
- Script.get_rpc_methods renamed to Script.get_rpc_config.
- Node.rpc[_id] and Callable.rpc now return an Error.
- Refactor MultiplayerAPI to allow extension.
- New MultiplayerAPI.rpc method with Array argument (for scripts).
- Move the default MultiplayerAPI implementation to a module.
Implement built-in classes Vector4, Vector4i and Projection.
* Two versions of Vector4 (float and integer).
* A Projection class, which is a 4x4 matrix specialized in projection types.
These types have been requested for a long time, but given they were very corner case they were not added before.
Because in Godot 4, reimplementing parts of the rendering engine is now possible, access to these types (heavily used by the rendering code) becomes a necessity.
**Q**: Why Projection and not Matrix4?
**A**: Godot does not use Matrix2, Matrix3, Matrix4x3, etc. naming convention because, within the engine, these types always have a *purpose*. As such, Godot names them: Transform2D, Transform3D or Basis. In this case, this 4x4 matrix is _always_ used as a _Projection_, hence the naming.
- Add support for explicit values in properties using `PROPERTY_HINT_FLAGS`
that works the same way it does for enums.
- Fix enums and flags in VisualScriptEditor (it wasn't considering the
explicit value).
- Use `PROPERTY_HINT_FLAGS` for C# enums with the FlagsAttribute instead
of `PROPERTY_HINT_ENUM`.
* Map is unnecessary and inefficient in almost every case.
* Replaced by the new HashMap.
* Renamed Map to RBMap and Set to RBSet for cases that still make sense
(order matters) but use is discouraged.
There were very few cases where replacing by HashMap was undesired because
keeping the key order was intended.
I tried to keep those (as RBMap) as much as possible, but might have missed
some. Review appreciated!
Adds a new, cleaned up, HashMap implementation.
* Uses Robin Hood Hashing (https://en.wikipedia.org/wiki/Hash_table#Robin_Hood_hashing).
* Keeps elements in a double linked list for simpler, ordered, iteration.
* Allows keeping iterators for later use in removal (Unlike Map<>, it does not do much
for performance vs keeping the key, but helps replace old code).
* Uses a more modern C++ iterator API, deprecates the old one.
* Supports custom allocator (in case there is a wish to use a paged one).
This class aims to unify all the associative template usage and replace it by this one:
* Map<> (whereas key order does not matter, which is 99% of cases)
* HashMap<>
* OrderedHashMap<>
* OAHashMap<>
These typedefs don't save much typing compared to the full `Ref<Resource>`
and `Ref<RefCounted>`, yet they sometimes introduce confusion among
new contributors.
* Very old macros from the time Godot was created.
* Limited arguments to 5 (then later changed to 8) in many places.
* They were replaced by C++11 Variadic Templates.
* Renamed methods that take argument pointers to have a "p" suffix. This was used in some places and not in others, so made it standard.
* Also added a dereference check for Variant*. Helped catch a couple of bugs.
When reloading C# classes and keep their properties values they are
retrieved and stored in a state list.
Retrieving the properties was only getting the fields of the C# class
and not inherited fields so those properties values were lost on reload.
Now we also try to find the field in the parent classes.
Ensures that the `get_property_list` and `get_script_property_list`
methods push the script properties to the end of the given list, this
prevents the script property from appearing after the script variables.
Sets `AlignOperands` to `DontAlign`.
`clang-format` developers seem to mostly care about space-based indentation and
every other version of clang-format breaks the bad mismatch of tabs and spaces
that it seems to use for operand alignment. So it's better without, so that it
respects our two-tabs `ContinuationIndentWidth`.
Apply suggestions from code review
Merging @akien-mga's suggestion with the matching change to the CS project
Co-authored-by: Rémi Verschelde <rverschelde@gmail.com>
- implement CSharpInstance::get_method_list
- loop through parent classes in CSharpInstance::get_method_list and CSharpScript::get_script_method_list (#46408)
Move multiplayer classes to "core/multiplayer" subdir.
Move the RPCConfig and enums (TransferMode, RPCMode) to a separate
file (multiplayer.h), and bind them to the global namespace.
Move the RPC handling code to its own class (RPCManager).
Renames "get_rpc_sender_id" to "get_remote_sender_id".
This commit completely removes the RPC_MODE_MASTER ("master" keyword),
and renames the RPC_MODE_PUPPET to RPC_MODE_AUTHORITY ("auth" keyword).
This commit also renames the "Node.[get|set]_network_master" methods to
"Node.[get|set]_network_authority".
This commit also renames the RPC_MODE_REMOTE constant to RPC_MODE_ANY.
RPC_MODE_MASTER in Godot 3.x meant that a given RPC would be callable by
any puppet peer on the master, while RPC_MODE_PUPPET meant that it would
be callable by the master on any puppet.
Beside proving to be very confusing to the user (referring to where it
could be called instead of who can call it) the RPC_MODE_MASTER is quite
useless. It is almost the same as RPC_MODE_REMOTE (anyone can call) with
the exception that the network master cannot. While this could be useful
to check in some case, in such a function you would anyway need to check
in code who is the caller via get_rpc_sender_id(), so adding the check
there for those rare cases does not warrants a dedicated mode.
- Move the "sync" property for RPCs to RPCConfig.
- Unify GDScript annotations into a single one:
- `@rpc(master)` # default
- `@rpc(puppet)`
- `@rpc(any)` # former `@remote`
- Implement three additional `@rpc` options:
- The second parameter is the "sync" option (which also calls the
function locally when RPCing). One of "sync", "nosync".
- The third parameter is the transfer mode (reliable, unreliable,
ordered).
- The third parameter is the channel (unused for now).
* The harcoded 8 slots are no more and impose limits in the new extension system.
* New system is limitless, although it will impose small performance hit with a mutex.
* Use a token to request the instance binding.
**Warning**: Mono will most likely break as a result of this, will need to be modified to use the new system.
- Fix C++ compile errors about pending variable renames after the `Reference` to `RefCount` change.
- Fix C# compile errors due to the recent rename of `EnablePlugin()` and `Build()`, which are now underscore-prefixed in bindings.
- Additional rename: `godot_icall_Reference_Dtor` to `godot_icall_RefCounted_Dtor`.
In this PR:
- Removed rset
- rpc_config can now optionally configure transfer mode
(reliable/unreliable/ordered) and channel (channels are not actually
implemented yet.)
- Refactor how the RPC id is computed to minimize the logic in Node and
scripts that now only needs a single `get_rpc_methods` function.
The current code style guidelines forbid the use of `auto`.
Some uses of `auto` are still present, such as in UWP code (which
can't be currently tested) and macros (where removing `auto` isn't
easy).
This source generator adds a newly introduced attribute,
`ScriptPath` to all classes that:
- Are top-level classes (not inner/nested).
- Have the `partial` modifier.
- Inherit `Godot.Object`.
- The class name matches the file name.
A build error is thrown if the generator finds a class that meets these
conditions but is not declared `partial`, unless the class is annotated
with the `DisableGodotGenerators` attribute.
We also generate an `AssemblyHasScripts` assembly attribute which Godot
uses to get all the script classes in the assembly, eliminating the need
for Godot to search them. We can also avoid searching in assemblies that
don't have this attribute. This will be good for performance in the
future once we support multiple assemblies with Godot script classes.
This is an example of what the generated code looks like:
```
using Godot;
namespace Foo {
[ScriptPathAttribute("res://Player.cs")]
// Multiple partial declarations are allowed
[ScriptPathAttribute("res://Foo/Player.cs")]
partial class Player {}
}
[assembly:AssemblyHasScripts(new System.Type[] { typeof(Foo.Player) })]
```
The new attributes replace script metadata which we were generating by
determining the namespace of script classes with a very simple parser.
This fixes several issues with the old approach related to parser
errors and conditional compilation.
It also makes the task part of the MSBuild project build, rather than
a separate step executed by the Godot editor.
-Added a new method in Resource: reset_state , used for reloading the same resource from disk
-Added a new cache mode "replace" in ResourceLoader, which reuses existing loaded sub-resources but resets their data from disk (or replaces them if they chaged type)
-Because the correct sub-resource paths are always loaded now, this fixes bugs with subresource folding or subresource ordering when saving.
Happy new year to the wonderful Godot community!
2020 has been a tough year for most of us personally, but a good year for
Godot development nonetheless with a huge amount of work done towards Godot
4.0 and great improvements backported to the long-lived 3.2 branch.
We've had close to 400 contributors to engine code this year, authoring near
7,000 commit! (And that's only for the `master` branch and for the engine code,
there's a lot more when counting docs, demos and other first-party repos.)
Here's to a great year 2021 for all Godot users 🎆
The underscore prefix was used to avoid the conflict between the `RID` class
name and the matching enum value in `Variant::Type`.
This can be fixed differently by prefixing uses of the `RID` class in `Variant`
with the scope resolution operator, as done already for `AABB`.
Removes code duplication between `CSharpScript::reload()` and `CSharpScript::initialize_for_managed_type()`.
Removes a redundant `CSharpScript::update_exports()` call in `CSharpLanguage::reload_assemblies()` as `CSharpScript::reload()` already calls it when appropriate.
Fixes missing update of RPC information in `CSharpScript::initialize_for_managed_type()`.
Godot.NET.Sdk
-------------
Godot uses its own custom MSBuild Sdk for game
projects. This new Sdk adds its own functionality
on top of 'Microsoft.NET.Sdk'.
The new Sdk is resolved from the NuGet package.
All the default boilerplate was moved from game
projects to the Sdk. The default csproj for
game project can now be as simple as:
```
<Project Sdk="Godot.NET.Sdk/4.0.0-dev2">
<PropertyGroup>
<TargetFramework>netstandard2.1</TargetFramework>
</PropertyGroup>
</Project>
```
Source files are included by automatically so
Godot no longer needs to keep the csproj in sync
when creating new source files.
Define constants
----------------
Godot defines a list of constants for conditional
compilation. When exporting games, this list also
included engine 'features' and platform 'bits'.
There were a few problems with that:
- The 'features' constants were only defined when
exporting games. Not when building the game for
running in the editor player.
- If the project was built externally by an IDE,
the constants wouldn't be defined at all.
The new Sdk assigns default values to these
constants when not built from the Godot editor,
i.e.: when built from an IDE or from the command
line. The default define constants are determined
from the system MSBuild is running on.
However, it's not possible for MSBuild to
determine the set of supported engine features.
It's also not possible to determine if a project
is being built to run on a 32-bit or 64-bit
Godot executable.
As such the 'features' and 'bits' constants had
to be removed.
The benefit of checking those at compile time
was questionable, and they can still be checked
at runtime.
The new list of define constants includes:
- GODOT
- GODOT_<PLATFORM>
Defaults to the platform MSBuild is running on.
- GODOT_<PC/MOBILE/WEB>
- TOOLS
When building with the 'Debug' configuration
(editor and editor player).
- GODOT_REAL_T_IS_DOUBLE
Not defined by default unless $(GodotRealTIsDouble)
is overriden to be 'true'.
.NET Standard
-------------
The target framework of game projects was changed
to 'netstandard2.1'.
In general they are more confusing to users because they expect
inheritance to fully override parent methods. This behavior can be
enabled by script writers using a simple super() call.
Any C# file can be loaded as script and at load
time we don't yet know if it's actually meant to
be used as a script. As such, such an check can
result in a lot of false errors.
If the file is really meant to be used as a
script, an error would be printed later when
attempting to instantiate it any way.
Which means that reduz' beloved style which we all became used to
will now be changed automatically to remove the first empty line.
This makes us lean closer to 1TBS (the one true brace style) instead
of hybridating it with some Allman-inspired spacing.
There's still the case of braces around single-statement blocks that
needs to be addressed (but clang-format can't help with that, but
clang-tidy may if we agree about it).
Part of #33027.
When a child scene inherits a parent scene with a C# root node, the
parent scene's export variables appear to assume values set in the
parent scene, in the child scene's Inspector. However, when the child
scene is played, the parent scene's export variables assume default
values.
When a node is created, it inherits its parent C# script's fields from
the map CSharpScriptInstance::script->member_info. However this map was
not initialized outside the editor, and this commit ensured it is. This
fixes issues #36480 and #37581.
Note: Only replaced 2 instances to test, Node.get_children and TileMap.get_used_cells
Note: Will do a mass replace on later PRs of whathever I can find, but probably need
a tool to grep through doc.
Warning: Mono will break, needs to be fixed (and so do TypeScript and NativeScript, need to ask respective maintainers)
Also added an easier way to load native GLSL shaders.
Extras:
Had to fix no-cache for subresources in resource loader, it was not properly working, making shaders not properly reload.
Note:
The precommit hooks are broken because they don't seem to support enums from one class being used in another.
Feel free to fix this after merging this PR.
Right now, games only work on devices when exported with FullAOT+Interpreter.
There are some issues left that need to addressed for FullAOT alone. Right now,
it's giving issues with the Godot.NativeCalls static constructor.
Don't store GC handles for C# script instances and instance bindings as 'Ref<MonoGCHandle>'; store the raw data instead. Initially this was not possible as we needed to store a Variant, but this had not been the case for a looong time yet the stored type was never updated.
Implementation for new Variant types Callable, Signal, StringName.
Added support for PackedInt64Array and PackedFloat64Array.
Add generation of signal members as events, as well as support for
user created signals as events.
NOTE: As of now, raising such events will not emit the signal. As such,
one must use `EmitSignal` instead of raising the event directly.
Removed old ThreadLocal fallback class. It's safe to use thread_local now since
it's supported on all minimum versions of compilers we support.
For us, it practically only changes the fact that `A<A<int>>` is now
used instead of the C++03 compatible `A<A<int> >`.
Note: clang-format 10+ changed the `Standard` arguments to fully
specified `c++11`, `c++14`, etc. versions, but we can't use `c++17`
now if we want to preserve compatibility with clang-format 8 and 9.
`Cpp11` is still supported as deprecated alias for `Latest`.
EngineDebugger is the new interface to access the debugger.
It tries to be as agnostic as possible on the data that various
subsystems can expose.
It allows 2 types of interactions:
- Profilers:
A subsystem can register a profiler, assigning it a unique name.
That name can be used to activate the profiler or add data to it.
The registered profiler can be composed of up to 3 functions:
- Toggle: called when the profiler is activated/deactivated.
- Add: called whenever data is added to the debugger
(via `EngineDebugger::profiler_add_frame_data`)
- Tick: called every frame (during idle), receives frame times.
- Captures: (Only relevant in remote debugger for now)
A subsystem can register a capture, assigning it a unique name.
When receiving a message, the remote debugger will check if it starts
with `[prefix]:` and call the associated capture with name `prefix`.
Port MultiplayerAPI, Servers, Scripts, Visual, Performance to the new
profiler system.
Port SceneDebugger and RemoteDebugger to the new capture system.
The LocalDebugger also uses the new profiler system for scripts
profiling.
Main:
- It's now implemented thanks to `<mutex>`. No more platform-specific implementations.
- `BinaryMutex` (non-recursive) is added, as an alternative for special cases.
- Doesn't need allocation/deallocation anymore. It can live in the stack and be part of other classes.
- Because of that, it's methods are now `const` and the inner mutex is `mutable` so it can be easily used in `const` contexts.
- A no-op implementation is provided if `NO_THREADS` is defined. No more need to add `#ifdef NO_THREADS` just for this.
- `MutexLock` now takes a reference. At this point the cases of null `Mutex`es are rare. If you ever need that, just don't use `MutexLock`.
- Thread-safe utilities are therefore simpler now.
Misc.:
- `ScopedMutexLock` is dropped and replaced by `MutexLock`, because they were pretty much the same.
- Every case of lock, do-something, unlock is replaced by `MutexLock` (complex cases where it's not straightfoward are kept as as explicit lock and unlock).
- `ShaderRD` contained an `std::mutex`, which has been replaced by `Mutex`.
- Renames PackedIntArray to PackedInt32Array.
- Renames PackedFloatArray to PackedFloat32Array.
- Adds PackedInt64Array and PackedFloat64Array.
- Renames Variant::REAL to Variant::FLOAT for consistency.
Packed arrays are for storing large amount of data and creating stuff like
meshes, buffers. textures, etc. Forcing them to be 64 is a huge waste of
memory. That said, many users requested the ability to have 64 bits packed
arrays for their games, so this is just an optional added type.
For Variant, the float datatype is always 64 bits, and exposed as `float`.
We still have `real_t` which is the datatype that can change from 32 to 64
bits depending on a compile flag (not entirely working right now, but that's
the idea). It affects math related datatypes and code only.
Neither Variant nor PackedArray make use of real_t, which is only intended
for math precision, so the term is removed from there to keep only float.
Issues caused by cf8c679a23.
The Mono change is actually a bugfix (used the int instead of ObjectID
by mistake).
The GDNative change is a temporary revert until a more exhaustive approach
is taken to make 'godot_int' 64-bit, is confirmed wanted by GDNative users.
- Now is sent the method ID rather the full function name.
- The passed IDs (Node and Method) are compressed so to use less possible space.
- The variant (INT and BOOL) is now encoded and compressed so to use much less data.
- Optimized RPCMode retrieval for GDScript functions.
- Added checksum to assert the methods are the same across peers.
This work has been kindly sponsored by IMVU.
As pointed out by Faless, a do{ } while(0) wrapper around a continue or
break just ends the do{ } while(0) loop. The do{ } while(0) loop exists
to enable the macro to be used as a function which requires a semicolon.
The alternative approach is to use an if(1) { } else ((void)0) wrapper.
Since the macro already has an if(unlikely(m_cond)) { } this patch simply
adds the else ((void)0) to this if statement instead.
For consistency all the macros have been updated in the same way, and
trailing else warnings corrected. However, the wrappers around ERR_PRINT
and WARN_PRINT were removed, because they generated too many ambiguous
trailing else warnings. They are also single line macros so a wrapper is
not needed.
The code was attempting to dynamic cast the native instance to Reference after
the managed instance was disposed. As the managed instance acts as a Ref,
the native instance was freed during that disposal.
This made the dynamic cast fail and we attempted to memdelete a second time.
The fix is to make the dynamic cast before disposal.
Because of the weird case with multi-threading and ResourceLoader, it can be the case that a resource is GCed while being referenced again in the main thread. In such cases, a new unsafe reference is created before the finalizer thread removes the previous one.
- `EditorNavigationMeshGenerator` was being registered as part of the Core API,
even after d3f48f88bb. We must make sure to
set Editor as the current ClassDB API type before creating an instance.
- The `VisualScriptEngineSingleton.constant` property has a property hint string
that's different between tools and non-tools builds. This commit makes the
hint string to no longer be set in `_bind_methods`, and to instead set it in
`_validate_property`. This way it's ignored when calculating the API hash.
- `JavaClassWrapper` is now registered in ClassDB on all platforms,
using a dummy implementation on platforms other than Android.
This fixes API portability between Android and other platforms.
- Updated `--class-db-json` command to ignore non-virtual methods that start
with an underscore (see: 4be87c6016).
Added guards to all C# script interface calls to attach the current thread
for the current scope if the thread is not already attached.
This is far from ideal, as attaching the thread is not cheap and all managed
thread local storage is lost when we detach the thread at the end of the calls.
However, it's the best we can do for now to avoid crashing
when an unattached thread tries to interact with C# code.
Happy new year to the wonderful Godot community!
We're starting a new decade with a well-established, non-profit, free
and open source game engine, and tons of further improvements in the
pipeline from hundreds of contributors.
Godot will keep getting better, and we're looking forward to all the
games that the community will keep developing and releasing with it.
VarArg methods have the return type Object in the API json for GDNative. This
can cause undefined behavior in some language bindings due to lack of
documentation on VarArg methods' behavior.
This changes the MethodInfo of:
- CSharpScript::_new
- GDScript::_new
- PluginScript::_new
