Co-authored-by: Gordon MacPherson <gordon@gordonite.tech>
Co-authored-by: Andrea Catania <info@andreacatania.com>
Co-authored-by: K. S. Ernest (iFire) Lee <ernest.lee@chibifire.com>
This is a complete rewrite of the importer. It will give more deterministic behaviour and has been sponsored by IMVU inc, over 1 year has gone into the development of this importer to remove the burden of the FBX SDK.
This was my project for 1 entire year and I really enjoyed the opportunity to add to Godot.
Along the road of implementing fixes we implemented fbx pivots, animations and inheritance type handling, which in most cases works properly.
We have implemented animation and mesh skinning too this should work out of the box, if there are issues let us know.
It's designed so that you can expand this with ease, and fix bugs easily too.
It can import from Autodesk Maya and import into Godot, with pivots.
There are bits we could polish but for now this is good enough.
Additional fixes made before upstreaming:
- fixed memory leaks
- ensure consistent ordering on mac linux and windows for fbx tree. (very important for material import to be deterministic)
- disabled incorrect warnings for fbx_material
- added compatibility code for /RootNode/ so compat is not broken
- Optimise FBX - directly import triangles
- remove debug messages
- add messages for mesh id, mesh re-import is sometimes slow and we need to know what mesh is being worked on
- Document no longer uses unordered maps
- Removed some usages of &GetRequiredToken replaced with safe *GetRequiredToken() function
- Added parser debugging
- Added ERR_FAIL_CONDS for unsupported mesh formats (we can add these later super easy to do now)
- Add memory debugging for the Tokens and the TokenParser to make it safe
- Add memory initialisation to mesh.cpp surface_tool.h and mesh.h
- Initialise boolean flags properly
- Refactored to correct naming for the fbx_mesh_data.h so you know what data you are working on
- Disabled corruption caused by the FIXME:
- Fixed document reading indexes and index_to_direct vs indexes mode
- Fixed UV1 and UV2 coordinates
- Fixed importer failing to import version 7700 files
- Replaced memory handling in the FBX Document with pointers, before it was dereferencing invalid memory.
- Fixed typed properties
- Improved Document API
- Fixed bug with ProcessDOMConnection() not working with the bool flag set to true.
- Fixed FBX skinning not deforming for more than one single mesh
- Fixed FBX skeleton mapping and skin mapping not being applied properly (now retrieved from document skin list)
- Fixed set_bone_pose being used in final version()
- Fixed material properties exceeding 1.0.
- FBX Document parser revamped to use safe memory practices, and with graceful error messages.
- ScopePtr, TokenPtr and various internal types have been fleshed out to use proper typedefs across the codebase.
- Fixed memory leaks caused by token cleanup failing (now explicit cleanup step, no shared_ptr, etc)
- Fixed bug with PropertyTable not reading all properties and not cleaning up properly.
- Fixed smoothing groups not working
- Fixed normal duplications
- Fixed duplication check for pre-existing coordinates.
- Fixed performance of vertex lookup in large meshes being slow, using lookup table separate to the data for indexing, this reduces import time from 10 minutes of bistro down to 30 seconds.
- Fixed includes requiring absolute path in headers and cpp files using CPPPath.
Bugs/Features wish list:
- locator bones
- quat anim key interpolation (most fbx maya files have euler rotations from blender and maya, nobody uses this)
- some rigs skins scale up when SSC enabled inconsistently per bone
- some skins can disappear entirely
- material mapping needs expanded, but this will be done for 4.0 as it requires rewrite.
Workarounds for issues found until we patch them:
- mesh -> clear skin can resolve most of the bugs above.
- locators can be worked around by removing them before exporting your rig.
- some material properties wont always import, this is okay to override in the material properties.
**If you are having issues or need support fear not!**
Please provide minimal rigs which can reproduce issues as we can't spend a lot of time investigating each rig. We need a small example which breaks and we can then sort the problem. In some cases this is not possible so its okay to privately send models to us via IRC or a ticket and we can provide an email address, we won't reveal or disclose privately sent rig files to any companies, or to companies I work for, they will not be shared, only tested and bugs will be drawn up from the conclusions. Also include identifying information about what you did and how it didn't work. Please file each file separately in a bug report, unless the problem is the same.
This was sponsored by IMVU, and a special thanks to everyone who supported this project.
Signed-off-by: Gordon MacPherson <gordon@gordonite.tech>
This is needed because of the new changes to Variant. The reference
counter is increased by adding it to a Variant, which means no GDScript
will be freed (or will be double freed if manually freed somewhere).
(cherry picked from commit 4d960efafc)
It's not an engine module. This is handled by GDNative's
`SCsub` instead, as done for other subdirectories already.
(cherry picked from commit 650ae413ce)
Continuing the work from f43a0ef327,
It seems the maps visibility was not actually set and a flase value was propogated to the meshes
Trying to set the maps visibility directly (Using `set_visibility` causes the map to no longer to recieve visibility notifications, instead this approach was chosen
(cherry picked from commit fe52c6b0b7)
- Removed item list that displayed multiple build
configurations launched. Now we only display
the last build that was launched.
- Display build output next to the issues list.
Its visibility can be toggled off/on.
This build output is obtained from the MSBuild
process rather than the MSBuild logger. As such
it displays some MSBuild fatal errors that
previously couldn't be displayed.
- Added a context menu to the issues list with
the option to copy the issue text.
- Replaced the 'Build Project' button in the panel
with a popup menu with the options:
- Build Solution
- Rebuild Solution
- Clean Solution
- The bottom panel button was renamed from 'Mono'
to '.NET' and now display an error/warning icon
if the last build had issues.
The GDScript `load` mention is moved from the class `ResourceLoader`
description to the `ResourceLoader.load` method description instead,
where it is more likely to be found.
(cherry picked from commit a8404cf56c)
Modify usage of types so that the `Ref` created from `base_type.script_type` doesn't involve converting first to `Variant`, which will use the constructor for `Object *`, as if the argument wasn't a `Reference`, and therefore will convert back to null.
without this we have to manually drop a `x86_64` builds of gdnative libs into the Xcode project to allow running in `iOS Simulator`
(cherry picked from commit f49199bcc3)
Fixes: #28683, #28621, #28596 and maybe others
For iOS we enable pvrtc feature by default for both GLES2/GLES3
Etc1 for iOS doesn't have any sense, so it disabled.
Fixed checks in export editor.
Fixed pvrtc ability detection in GLES2 driver.
Fixed pvrtc encoding procedure.
When generating certificates with
`Crypto.generate_self_signed_certificate` we generate the PEM in a
buffer via `mbedtls_x509write_crt_pem`.
Since version 2.16.8, mbedtls adds spurious data at the end of the
buffer due to internal optimizations, this breaks our logic when we try
to immediately parse it and return a proper `X509Certificate` object.
This commit updates the code to find the actual PEM length to parse
using `strlen`, takes extra caution always adding the terminator to the
buffer, and slightly improve error messages.
(cherry picked from commit 60687ce778)
When NormalizePath was called with an absolute
path (with drive letter) on Windows, it would
prepend a file path separator to the path, e.g.:
'\C:\Program Files\'.
Apparently this was still accepted as a valid
path by DotNetGlob and it stopped working when
we switched to MSBuildGlob.
(cherry picked from commit 1db0395950)
The previous max worked OK for audio data, but stb_vorbis recently gained support
for Vorbis comments, which can embed up to 2^32-1 bytes of data (e.g. cover art
encoded as base64).
We use 2^30 as max which should be sufficient for most files.
Fixes#41913.
(cherry picked from commit d16f5a57c1)
At least on Windows there seems to be issues if
the solution has no BOM and contains a project
with cyrillic chars.
(cherry picked from commit 1c74fa4242)
MSBuild Item returns empty strings if an attribute isn't set (which
caused an IndexOutOfRangeException in NormalizePath).
We were treating Excludes incorrectly, Remove directives provide the
intended behaviour in the auto-including csproj format.
This error was normally being printed when
trying to open the project assembly while
the project was not yet built.
The error should not be printed. It's the job
of this method's caller to decide whether to
print an error or not if loading failed.
The editor wasn't clearing the debugger agent
settings properly after a processing a play
request from an IDE. This caused consequent play
attempts to fail if not launched from the IDE,
as the game would still attempt and fail to
connect to the debugger.
The concrete cause: Forgetting to clear the
`GODOT_MONO_DEBUGGER_AGENT` environment variable.
(cherry picked from commit 6e7da72648)
This is a cherry-pick of
ced77b1e9b
with several 3.2 specific alterations.
There are a lot of build issues coming from
old style projects. At this point fixing every
single one of those would require adding patch
after patch to the project file, which is a
considerable amount work and makes the csproj
even more bloated than it already is.
As such I decided this effort would be better
spent back-porting the Sdk style support that's
already available in 4.0-dev to the 3.2 branch.
This will prevent many issues, but it will also
introduce other benefits, among them:
- While target framework stays as .NET Framework
v4.7.2, it can be changed to .NET Standard 2.0
or greater if desired.
- It makes it much easier to add future patches.
They are added to Godot.NET.Sdk and the only
change required in Godot code is to update the
Sdk version to use.
- Default Godot define constants are also
backported, which fixes IDE issues with the
preprocessor.
There are a few differences in the changes
applied during patching of the csproj compared
to 4.0 with the purpose of preventing breaking
builds:
- 'TargetFramework' stays net472 both for new
projects and when importing old ones. It can
be manually changed to netstandard 2.0+ if
desired though.
The following features are enabled by default for
new projects. Enabling them in imported projects
may result in errors that must be fixed manually:
- 'EnableDefaultCompileItems' is disabled as it
can result in undesired C# source files being
included. Existing include items are kept.
As long as 'EnableDefaultCompileItems' remains
disabled, Godot will continue taking care of
adding and removing C# files to the csproj.
- 'GenerateAssemblyInfo' is disabled as it
guarantees a build error because of conflicts
between the existing 'AssemblyInfo.cs' and the
auto-generated one.
- 'Deterministic' is disabled because it doesn't
like wildcards in the assembly version (1.0.*)
that was in the old 'AssemblyInfo.cs'.
Of importance:
This is a breaking change. A great effort was
put in avoiding build errors after upgrading a
project, but there may still be exceptions.
This also breaks forward compatibility. Projects
opened with Godot 3.2.3 won't work out of the box
with older Godot versions. This was already the
case with changes introduced in 3.2.2.
Albeit C# support in 3.2.x was still labeled as
alpha, we've been trying to treat it as stable
for some time. Still the amount of problems this
change solves justifies it, but no more changes
that break project compatibility are to be
introduced from now on (at least for 3.x).
This upgrade is needed in order to support
reading and editing project files that use Sdks
as well as other new features. A common example
in 3.2 is having to specify a PackageReference
version with a child element rather than the
attribute. This is no longer the case now.
Partial cherry-pick of f3bcd5f8dd
Most of the other changes from that commit were already partially
cherry-picked in 3928fe200f.
Not sure if we should check revision too, but this is good enough for what we want.
This will be needed to load the correct Microsoft.Build when we switch to the nuget version.
Manual cherry-pick of af4acb5b11 (relevant parts)
This change avoids the editor to freeze for several seconds when a
project with lots of scripts is loaded in the editor.
It focuses on a few heavy operations previously executed on all
previously loaded scripts:
- Initialize script resource (script validation/parsing) only
on focus
- ScriptTextEditor: code editor and edit menu are added to the
scene only on focus
- Add to recent scripts only when opening new scripts
(load/save scene metadata)
My initial attempt changed this in the gdscript code, which resulted in
a duplicate warning name in the builtin editor. We should just append
the warning name in the LSP instead.
This uses parens to match what is shown in the builtin editor.
(cherry picked from commit 8dcc39ec91)
Occasionally you want to ignore a warning with a `warning-ignore`
comment, and you have to go into the settings to look up what the
actual name of the warning is. This patch appends the warning name to
the end of the warning so you know what string to use to ignore it,
similar to other linters like pylint.
For example
```
"The signal 'blah' is declared but never emitted.";
```
is now
```
"The signal 'blah' is declared but never emitted. (UNUSED_SIGNAL)";
```
(cherry picked from commit de3ad3b30e)
fix crash when pass null in print array in GD.print 2
fix crash when pass null in print array in GD.print 3
fix space
(cherry picked from commit d2461bad63)
For some reason `mono_unhandled_exception` is not
printing the exception as its comment claims.
Use `mono_print_unhandled_exception` instead.
(cherry picked from commit f87ae395ee)
Sometimes Visual Studio documents have the root path all in upper case.
Since Godot doesn't support loading resource files with a case insensitive path,
this makes script resource loading to fail when the Godot editor gets code
completion requests from Visual Studio.
This fix allows the resource path part of the path to be case insensitive. It
still doesn't support cases where the rest of the path is also case insensitive.
For that we would need a proper API for comparing paths. However, this fix
should be enough for our current cases.
MSDN says:
> When you write .dds files, you should set the DDSD_CAPS and
> DDSD_PIXELFORMAT flags, and for mipmapped textures you should also
> set the DDSD_MIPMAPCOUNT flag. However, when you read a .dds file,
> you should not rely on the DDSD_CAPS, DDSD_PIXELFORMAT, and
> DDSD_MIPMAPCOUNT flags being set because some writers of such a file
> might not set these flags.
https://docs.microsoft.com/en-us/windows/win32/direct3ddds/dds-header
¯\_(ツ)_/¯
Fixes#39516.
(cherry picked from commit 0f21249a2d)
Reverts `latest_client_id` back to 0, as I misunderstood how the client
IDs are assigned and, without further testing and debugging, I can't
say if this was a bug or a valid default value.
Similarly, a `latest_client_id` of -1 is no longer raising an error.
Fixes#39548.
(cherry picked from commit 786f4ada35)
Build PCRE2 thirdparty library with unicode support. RegEx objects
in Godot can now be used to recognize unicode strings.
(cherry picked from commit 9f10f6779c)
Configured for a max line length of 120 characters.
psf/black is very opinionated and purposely doesn't leave much room for
configuration. The output is mostly OK so that should be fine for us,
but some things worth noting:
- Manually wrapped strings will be reflowed, so by using a line length
of 120 for the sake of preserving readability for our long command
calls, it also means that some manually wrapped strings are back on
the same line and should be manually merged again.
- Code generators using string concatenation extensively look awful,
since black puts each operand on a single line. We need to refactor
these generators to use more pythonic string formatting, for which
many options are available (`%`, `format` or f-strings).
- CI checks and a pre-commit hook will be added to ensure that future
buildsystem changes are well-formatted.
(cherry picked from commit cd4e46ee65)
`latest_client_id` now defaults to `-1` (invalid ID) instead of `0`.
Also fix typo in notification `gdscrip_client/changeWorkspace`,
and fix argument names in method binds.
Fixes#39375.
(cherry picked from commit e34f33711b)
ice_candidate_created should be emitted after set_local_description no
matter the type of the description (assuming no error is returned of
course).
(cherry picked from commit 39bcbf5690)
Attempting to move the function node to another function whose data connection is a dependency of the node the specific node being moved to a different function during changes to sequence connections.
By skipping, if the from_node is a function_node during the data connection dependencies scan.
Should fix#37991
(cherry picked from commit 5c48631509)
Simple fix for #38627. iOS (#20268) and HTML5 (#20270) removed from list of exceptions
for platforms supported in warning message.
(cherry picked from commit 3d03be7a56)
Depending on the conditional statements of the 'for' and 'while' loops,
their body may not even execute once. For example:
func a():
var arr = []
for i in arr:
return i
# can be reached, but analysis says cannot
return -1
func b():
var should_loop = false
while should_loop:
return 1
# can be reached, but analysis says cannot
return 0
The parser will complain that the statements after the comment cannot
be reached, but it is clearly possible for our scenario. This is
because the parser falsely assumes that the loop body will always
execute at least once.
Fix the code to remove this assumption for both of those loops.
(cherry picked from commit 7b1423a61e)
A class can't have multiple signals with the same name, but previously users
would not be alerted to a conflict while editing the script where it occurred.
Now a helpful error will appear in the editor during script parsing.
(cherry picked from commit 9e44739324)
This patch adds ability to include external, user-defined C++ modules
to be compiled as part of Godot via `custom_modules` build option
which can be passed to `scons`.
```
scons platform=x11 tools=yes custom_modules="../project/modules"
```
Features:
- detects all available modules under `custom_modules` directory the
same way as it does for built-in modules (not recursive);
- works with both relative and absolute paths on the filesystem;
- multiple search paths can be specified as a comma-separated list.
Module custom documentation and editor icons collection and generation
process is adapted to work with absolute paths needed by such modules.
Also fixed doctool bug mixing absolute and relative paths respectively.
Implementation details:
- `env.module_list` is a dictionary now, which holds both module name as
key and either a relative or absolute path to a module as a value.
- `methods.detect_modules` is run twice: once for built-in modules, and
second for external modules, all combined later.
- `methods.detect_modules` was not doing what it says on the tin. It is
split into `detect_modules` which collects a list of available modules
and `write_modules` which generates `register_types` sources for each.
- whether a module is built-in or external is distinguished by relative
or absolute paths respectively. `custom_modules` scons converter
ensures that the path is absolute even if relative path is supplied,
including expanding user paths and symbolic links.
- treats the parent directory as if it was Godot's base directory, so
that there's no need to change include paths in cases where custom
modules are included as dependencies in other modules.
(cherry picked from commit a96f0e98d7)
