Commit graph

1440 commits

Author SHA1 Message Date
Hugo Locurcio
0445ccf428
Fix Image CowData crash when baking large lightmaps
This switches to 64-bit integers in select locations of the Image
class, so that image resolutions of 16384×16384 (used by
lightmap texture arrays) can be used properly. Values that are larger
should also work.

VRAM compression is also supported, although most VRAM-compressed
formats are limited to individual slices of 16384×16384. WebP
is limited to 16383×16383 due to format limitations.
2024-07-19 16:04:30 +02:00
Rémi Verschelde
293c0f7646
Merge pull request #94526 from RandomShaper/wtp_rl_prize_prequel
Batch of fixes for WorkerThreadPool and ResourceLoader (safe set)
2024-07-19 11:11:03 +02:00
Pedro J. Estébanez
28a7a95531 ResourceLoader: Fix sync issues with error reporting
This is about not letting the resource format loader set the error code directly on the task anymore. Instead, it's stored locally and assigned only when it is right to do so.

Otherwise, other tasks may see an error code in the current one before it's state having transitioned to errored. While this, besides the technically true data race, may not be a problem in practice, it causes surprising situations during debugging as it breaks assumptions.
2024-07-19 10:00:41 +02:00
Fabio Alessandrelli
31454423ba [Core] Fix TypedArray encoding when full objects is disabled 2024-07-18 09:58:17 +02:00
Pedro J. Estébanez
5b5cdf2414 Fixup recent changes to threading concerns
ResourceLoader:
- Fix invalid tokens being returned.
- Remove no longer written `ThreadLoadTask::dependent_path` and the code reading from it.
- Clear deadlock hazard by keeping the mutex unlocked during userland polling.

WorkerThreadPool:
- Include thread call queue override in the thread state reset set, which allows to simplify the code that handled that (imperfectly) in the ResourceLoader.
- Handle the mutex type correctly on entering an allowance zone.

CommandQueueMT:
- Handle the additional possibility of command buffer reallocation that mutex unlock allowance introduces.
2024-07-16 11:03:02 +02:00
Pedro J. Estébanez
62d9ce6445 Re-add resource thread-safety measures
These deferring measures were added to aid threaded resource loading in being safe.

They were removed as seemingly unneeded, but it seems they are needed so resources involved in threaded loading interact with others only after "sync points".
2024-07-15 12:15:22 +02:00
Rémi Verschelde
247a481001
Fix a couple GCC 14 -Wmaybe-uninitialized warnings 2024-07-09 16:12:22 +02:00
Pedro J. Estébanez
a426479b3f ResourceLoader: Fix error message due to already-awaited tasks being re-awaited 2024-07-08 12:19:44 +02:00
Pedro J. Estébanez
f952d3956c ResourceLoader: Fixup management of thread-specific status
- Allows the message queue override to flush after loading each resource, which was the original intent.
- Removes a redundant call to mark the thread as safe-for-nodes.
2024-07-08 10:30:25 +02:00
Alvin Wong
5b3857e5f1 Fix UTF-8 misintepreted as Latin-1 when logging to file 2024-07-07 02:46:41 +08:00
Pedro J. Estébanez
ec61c5064c ResourceLoader: Support polling and get-before-complete on the main thread 2024-06-28 11:25:10 +02:00
Pedro J. Estébanez
c1391489e3 GDScript: Enhance handling of cyclic dependencies 2024-06-26 17:44:32 +02:00
Rémi Verschelde
e78dc83ee8
Merge pull request #93540 from RandomShaper/res_load_uncached
`ResourceLoader`: Fix handling of uncached loads
2024-06-24 13:20:24 +02:00
Pedro J. Estébanez
884d1da938 ResourceLoader: Fix handling of uncached loads
- `CACHE_MODE_IGNORE_DEEP` is checked in addition to `CACHE_MODE_IGNORE` to determine if a load is uncached. This avoids crashes in uncached loads due to prematurely freed load tasks.
- Cached load tasks are isolated (not registered in the task map ever). This avoids regular loads from reusing in-flight cached loads, which is not correct.
2024-06-24 11:25:57 +02:00
Rémi Verschelde
c53a4a7f64
Merge pull request #90069 from KoBeWi/file_suicide_prevention_measures
Don't allow copying file into its own path
2024-06-21 16:49:26 +02:00
Rémi Verschelde
74f9f12c71
Merge pull request #92320 from Hilderin/fix-importing-assets-with-csv
Fix reimporting assets with csv in the project
2024-06-21 13:54:35 +02:00
Rémi Verschelde
087ef4b942
Merge pull request #93124 from RandomShaper/skull_trio
`ResourceLoader`: Let the caller thread use its own message queue override
2024-06-14 17:13:38 +02:00
Rémi Verschelde
75eb8ad510
Merge pull request #93091 from RandomShaper/remove_false_safety
Remove no longer needed thread safety measures
2024-06-14 17:13:31 +02:00
Rémi Verschelde
08a21fda8d
Merge pull request #93082 from RandomShaper/fix_err_deadlock
ResourceLoader: Avoid deadlock when awaiting a loader thread that failed early
2024-06-14 17:13:28 +02:00
Pedro J. Estébanez
cc6f5d1a7a ResourceLoader: Let the caller thread use its own message queue override 2024-06-13 10:31:11 +02:00
Pedro J. Estébanez
21c03d1956 WorkerThreadPool: Fix thread message queue not restored after overridden in a task
Also, simplifies the thread override teardown in MessageQueue.
2024-06-13 10:31:08 +02:00
bruvzg
e651421905
[TextServer, GDExtension] Fix building text servers as GDExtension, expose new/changed low-level methods to GDExtension API. 2024-06-12 19:30:19 +03:00
Pedro J. Estébanez
b6994a414d Remove no longer needed thread safety measures
This is basically a revertion of commit 84b85d894c.
2024-06-12 18:10:11 +02:00
Pedro J. Estébanez
bdcceef0e2 ResourceLoader: Avoid deadlock when awaiting a loader thread that failed early 2024-06-12 13:49:37 +02:00
Hilderin
f1099ab943 Fix reimporting assets with csv in the project 2024-06-11 17:04:15 -04:00
Rémi Verschelde
4629845ead
Merge pull request #92718 from jsjtxietian/importer-duplicate
Fix same importer will be added multiple times in `get_importers_for_extension`
2024-06-11 11:51:06 +02:00
Rémi Verschelde
e4fa8543ea
Revert "Fix FileSystem dock won't show any file folders"
This reverts commit 72856d633a.

Fixes #93022.
2024-06-11 11:46:35 +02:00
Hilderin
72856d633a Fix FileSystem dock won't show any file folders 2024-06-10 19:54:37 -04:00
jsjtxietian
7552d242a3 Fix same importer will be added multiple times in get_importers_for_extension 2024-06-03 17:28:46 +08:00
Pedro J. Estébanez
f61c63e3a1 Avoid editor error reporting using resource loader thread's call queues 2024-05-31 09:42:37 +02:00
clayjohn
3b9d074fd7 Ensure MovieWriter output is in gamma space when using HDR 2D 2024-05-28 15:30:13 -07:00
A Thousand Ships
4ed62665a2
Replace .bind(...).call_deferred() with .call_deferred(...) 2024-05-27 13:29:57 +02:00
BlueCube3310
d260a2ba74 Optimize half and float image conversion 2024-05-24 11:01:20 +02:00
Rémi Verschelde
944b95e1a5
Merge pull request #91897 from RandomShaper/res_unreg_if_true
Add an identity check to resource unregistration from cache
2024-05-16 09:32:03 +02:00
kobewi
413c11357d Use Core/Scene stringnames consistently 2024-05-13 23:41:07 +02:00
Pedro J. Estébanez
b70afac286 Add an identity check to resource unregistration from cache
This is needed because resources loaded with CACHE_MODE_IGNORE still have path_cache set.
2024-05-13 11:55:50 +02:00
kobewi
a262d2d881 Add shorthand for using singleton string names 2024-05-11 18:53:08 +02:00
A Thousand Ships
a0dbdcc3ab
Replace find with contains/has where applicable
* Replaces `find(...) != -1` with `contains` for `String`
* Replaces `find(...) == -1` with `!contains` for `String`
* Replaces `find(...) != -1` with `has` for containers
* Replaces `find(...) == -1` with `!has` for containers
2024-05-08 12:37:42 +02:00
A Thousand Ships
955d5affa8
Reduce and prevent unnecessary random-access to List
Random-access access to `List` when iterating is `O(n^2)` (`O(n)` when
accessing a single element)

* Removed subscript operator, in favor of a more explicit `get`
* Added conversion from `Iterator` to `ConstIterator`
* Remade existing operations into other solutions when applicable
2024-05-04 16:08:55 +02:00
K. S. Ernest (iFire) Lee
f9b488508c
Add PackedVector4Array Variant type
Co-authored-by: A Thousand Ships <96648715+AThousandShips@users.noreply.github.com>
Co-authored-by: Rémi Verschelde <rverschelde@gmail.com>
2024-05-03 00:58:27 +02:00
Rémi Verschelde
28a18cf1b8
Merge pull request #88763 from BlueCube3310/image-is-compressed-static
Image: Add static `is_format_compressed` function.
2024-05-02 14:01:18 +02:00
bruvzg
fc948e87f6 Add symlink API support for Windows, expose symlink methods. 2024-04-28 19:59:34 +03:00
Fabio Alessandrelli
efccebd3db [Core] Use unztell64 in FileAccessZIP to ensure 64 bit return 2024-04-26 11:49:54 +02:00
Rémi Verschelde
dcdaa7d5a9
Merge pull request #90900 from Calinou/file-logging-strip-ansi-escape-codes
Strip ANSI escape codes from file logging
2024-04-24 18:55:10 +02:00
Rémi Verschelde
00cc0a3e8f
Merge pull request #90091 from ajreckof/Fix-load-subtask-not-being-registered-leading-to-false-progress-values-
Fix load subtask not being registered leading to false progress values.
2024-04-24 18:55:04 +02:00
Hugo Locurcio
459f14c098
Strip ANSI escape codes from file logging
Text editors cannot display ANSI escape codes, so these should be
stripped from log files to ensure readability.

Since this uses a regex, this works both for `print_rich()`
and manually inserted ANSI escape codes.

Co-authored-by: bruvzg <7645683+bruvzg@users.noreply.github.com>
2024-04-24 01:02:47 +02:00
Rémi Verschelde
659cf339bc
Merge pull request #90931 from aaronfranke/is-ascii-alphabet-char
Rename internal `is_ascii_char` to `is_ascii_alphabet_char`
2024-04-22 12:53:28 +02:00
Rémi Verschelde
504d01b61e
Merge pull request #90403 from bruvzg/file_resize
[FileAccess] Implement `resize` method.
2024-04-22 12:51:55 +02:00
A Thousand Ships
49ae632b7d
Fix import and saving related crashes
* Don't add empty mesh to result when importing obj files
* Check for null resources in `ResourceSaver`
2024-04-22 12:13:19 +02:00
Aaron Franke
b1f5e9fe3a
Rename internal is_ascii_char to is_ascii_alphabet_char 2024-04-20 02:36:41 -07:00
Rémi Verschelde
56dfed4efd
Merge pull request #90751 from dalexeev/core-marshalls-fix-objects-as-id-in-typed-arrays
Core: Fix binary serialization of objects in typed arrays
2024-04-17 10:59:01 +02:00
Danil Alexeev
27b94eb44c
Core: Fix binary serialization of objects in typed arrays
* Allow typed arrays to be encoded even if `p_full_objects` is `false`.
* Use `Array[EncodedObjectAsID]` instead of the encoded element type
  when decoding an array if `p_allow_objects` is `false`.
2024-04-16 16:19:10 +03:00
Yorick de Wid
61a2f5c534
FIX: Return if TCP is in STATUS_CONNECTED
If the `StreamPeerTCP` is polled and the TCP connection is `STATUS_CONNECTED` it should return after polling netsocket. Without `return` poll keeps calling `_sock->connect_to_host` and `connect()`.
2024-04-16 10:32:11 +02:00
Danil Alexeev
3c62f5fe5c
Core: Fix RefCounted handling in marshalls.cpp 2024-04-15 13:53:40 +03:00
ajreckof
ca020ad8cd Fix load subtask not being registered leading to false progress values. 2024-04-13 01:42:29 +02:00
bruvzg
88b3e68f93
[FileAccess] Implement resize method. 2024-04-12 19:20:49 +03:00
bruvzg
57fde36f31
[Unix / DirAccess] Fix removing directory symlinks with remove, ensure erase_contents_recursive is not following directory symlinks. 2024-04-12 10:06:49 +03:00
A Thousand Ships
64146cb7f3
[Core] Add iteration support to Array 2024-04-10 14:49:34 +02:00
Rémi Verschelde
bff6955459
Merge pull request #85024 from Jordyfel/moving-scripts-of-types-of-exported-typed-arrays
When saving a resource with an exported typed array, check whether the type is an external resource
2024-04-09 22:25:00 +02:00
Jordyfel
82de00a174 Type of exported typed array can be external resource 2024-04-09 15:28:42 +03:00
Fabio Alessandrelli
8addab785d [IO] Fix marshall decoding when script is NIL
We changed how scripts are binary serialized, and added a check to
ensure the new format is enforced, but there is still a case where the
old format (plain "prop"-"value" combo) is used, and that is when the
script is NIL.
2024-04-07 09:55:15 +02:00
Pylgos
c2be2dfbde Fix Resource::get_rid override not working in GDExtension 2024-04-06 01:27:16 +09:00
Rémi Verschelde
c8088c100f
Merge pull request #90038 from AThousandShips/load_path
[Core] Fix `ResourceLoader.load` cache with relative paths
2024-04-04 14:36:51 +02:00
Rémi Verschelde
3fdc169751
Merge pull request #89889 from BoThompson/compute_image_metrics-fix
Remove a redundant `MAX` comparison within a loop in `compute_image_metrics`.
2024-04-04 14:35:17 +02:00
Rémi Verschelde
ec5cae37d6
Merge pull request #78219 from dalexeev/core-typed-arrays-bin-serialization
Core: Add typed array support for binary serialization
2024-04-04 14:30:22 +02:00
kobewi
7b1b9b86ed Don't allow copying file into its own path 2024-03-31 13:53:36 +02:00
A Thousand Ships
0274877def
[Core] Fix ResourceLoader.load cache with relative paths
Paths were not simplified meaning that `res://foo.bar` was treated
differently from `./foo.bar` and similar
2024-03-30 12:50:08 +01:00
Bo Thompson
75d65195ae
Remove a redundant MAX comparison within a loop in compute_image_metrics 2024-03-28 22:23:15 +01:00
bruvzg
082b420c0a
Implement OS.execute_with_pipe method to run process with redirected stdio.
Implement `pipe://*` path handling for creation of named pipes.
2024-03-27 11:41:16 +02:00
ppphp
d4e1a74efb Add 'override' mark to ResourceFormat class 2024-03-25 04:06:34 +08:00
Rémi Verschelde
d5f944ff10
Merge pull request #89261 from paulloz/core/fix-script-reloading-outside-script-editor
Fix how scripts reload outside of ScriptEditor
2024-03-24 01:15:23 +01:00
Danil Alexeev
c30bec7fca
Core: Add typed array support for binary serialization 2024-03-20 11:37:41 +03:00
Paul Joannon
63674648fb
Fix how scripts reload outside of ScriptEditor 2024-03-18 10:29:06 +01:00
Rémi Verschelde
453485aede
Merge pull request #89270 from Repiteo/enforce-typename-in-templates
Enforce template syntax `typename` over `class`
2024-03-14 22:34:37 +01:00
Rémi Verschelde
0475011c66
Merge pull request #82084 from ogapo/pr/pck-cache-merge
Merge `uid_cache.bin` and `global_script_class_cache.cfg` after mounting PCKs
2024-03-11 14:01:16 +01:00
Rémi Verschelde
1c8ef9e252
Merge pull request #89251 from KoBeWi/fastpector
Speed up inspector updates for TileMap
2024-03-09 00:52:09 +01:00
Rémi Verschelde
baf75d048c
Merge pull request #87448 from nfrmtk/check-buffer-size
Add check for negative `buffer_size` inside `StreamPeerGZIP::start_(de)compression`
2024-03-09 00:52:01 +01:00
nfrmtk
da886006a0 add check for negative buffer_size inside start_(de)compression method. 2024-03-08 17:49:48 +03:00
Thaddeus Crews
9903e6779b
Enforce template syntax typename over class 2024-03-07 22:39:09 -06:00
kobewi
7319b612f3 Speed up inspector updates for TileMap 2024-03-07 18:01:30 +01:00
Ryan
42ce14043e
Expose scene unique id functionality in Resource 2024-03-07 14:53:58 +01:00
David Nikdel
5e6adb4a2d Merge uid_cache.bin and global_script_class_cache.cfg after mounting PCKs
fixes godotengine#82061
fixes godotengine#61556

Also, distinguish between main pack and DLC packs.
It's desirable to downloaded content to be as small as possible. This change avoids bloating non-main pack files with new versions of resources that are all read on startup and never used again. They have no effect if loaded after startup.
- project.godot/project.binary file
- extension_list.cfg
- app icon and boot_splash
- .ico and .icns files (these can still be opted in for DLC by listing them explicitly in the include filter)
2024-03-06 12:14:21 -05:00
bruvzg
bf8ec7b81b
[Export] Use relative file base offset for embedded PCK. 2024-03-06 14:02:01 +02:00
Rémi Verschelde
172b254687
Merge pull request #88664 from RandomShaper/res_load_modes
Enhance cache modes in resource loading
2024-02-29 13:54:07 +01:00
Rémi Verschelde
7462b1a0b2
Merge pull request #88561 from RandomShaper/res_load_safer
Improve thread safety of resource loading
2024-02-29 13:53:58 +01:00
BlueCube3310
8239aeec8e Image: Add static is_format_compressed function. 2024-02-28 19:42:06 +01:00
Pedro J. Estébanez
84b85d894c Improve thread safety of resource loading 2024-02-28 15:19:22 +01:00
Pedro J. Estébanez
5e144022e7 Enhance cache modes in resource loading
- Unify documentation, hoping to clear misconcepctions about about propagation of the cache mode across dependant loads.
- Clarify in docs that `CACHE_MODE_REPLACE` now also works on the main resource (from #87008).
- Add two recursive modes, counterparts of `CACHE_MODE_REPLACE` and `CACHE_MODE_IGNORE`, since it seems some need them (see #59669, #82830).
- Let resources, even loaded with one of the ignore-cache modes, get a path, which is useful for tools.
2024-02-26 14:59:04 +01:00
Pedro J. Estébanez
851f1d49d7 Remove spammy dev-mode message about load deadlock prevention 2024-02-23 14:54:03 +01:00
Rémi Verschelde
eac054f1cc
Merge pull request #87971 from bruvzg/font_no_image_dup
[TextServer / Font] Do not duplicate images to prevent unnecessary embedding.
2024-02-15 17:35:36 +01:00
Muller-Castro
a8bc9f3e78 Add const lvalue ref to core/* container parameters 2024-02-14 11:20:36 -03:00
Rémi Verschelde
dc99c8d4a4
Merge pull request #87008 from KoBeWi/cache_mode_replace_fixes²
Fix² behavior of ResourceFormatLoader `CACHE_MODE_REPLACE`
2024-02-13 23:43:18 +01:00
Rémi Verschelde
b457a30311
Merge pull request #87628 from YuriSizov/assets-bigger-better-errors
Improve error reporting in the asset library and in related types
2024-02-09 18:09:16 +01:00
A Thousand Ships
684752e75b
Replace error checks against size with is_empty 2024-02-09 12:50:15 +01:00
bruvzg
19f1d5afa9
[TextServer / Font] Do not duplicate images to prevent unnecessary embedding. 2024-02-05 10:38:32 +02:00
vittorioromeo
71e0082ab0 Extract 'SafeBinaryMutex' to separate header
This change simply extracts 'SafeBinaryMutex' from 'mutex.h' to
    'safe_binary_mutex.h', in an effort to reduce the compilation
    speed impact of including `mutex.h`.
2024-02-04 16:59:54 +01:00
scgm0
2dae53c316
Explicitly initialize all of FileAccess::create_func[ACCESS_MAX] 2024-01-31 10:46:14 +01:00
Rémi Verschelde
3dfedd69ea
Merge pull request #87711 from matthew1006/threaded-loading-progress-fix
Fix `ResourceLoader.load_threaded_get_status` returning `[0]` constantly in exported projects.
2024-01-30 17:48:11 +01:00
Rémi Verschelde
b65c495d6e
Merge pull request #86845 from RandomShaper/no_load_regress
Avoid regressing in progress reporting in resource load
2024-01-29 21:33:00 +01:00
Matthew Edwards
7565d1f3ab Push p_original_path into load_paths_stack and sub_tasks instead of p_path. 2024-01-29 17:27:57 +00:00