Upstream "mark-delay" change from flambda-backend. (details)
Hack to work around accounting problem: artificially catch-up work_counter at the start of any slice when it falls very far behind alloc_counter. (details)
Get rid of the @since attribute for the Uchar.hash function
Given that this function is present in the module since it was introduced, the convension is to not have any @since attribute at the function level, since the module-level one applies.
This commit reverts the addition of the attribute done in PR #13240, see commit 9585cfe264c4d9b4185f60ebfbbec3f8e54d1432.
Commit
55e9cd44ed34d6a62b83d1a608a15de709df5b1e
by david.allsopp
Harden the cloexec.ml test on Windows
On slower machines, it's possible for the main part of the test (in fdstatus_main.ml) to complete before the cloexec.ml has actually terminated as part of `Unix.execv` (this is an artefact of how the underlying CRT `execv` call is implemented on Windows).
Strictly on native Windows, cloexec.ml creates an additional dummy file and takes a write lock on it (with `Unix.lockf`). Before running, if this file exists, fdstatus_main.ml will also try to take a write lock on it, providing a way of fdstatus_main.ml being sure that its ancestor has fully terminated without actually having to know its PID.
Commit
78fc5774e856cfbe133750c66dfab48e3ab75452
by david.allsopp
Harden cloexec.ml test against virus scanners!
Particularly on slow VMs, on Windows it's possible to end up seeing a "permission denied" error trying to delete tmp.txt at the end of the test. The issue is that Windows Defender (or other virus scanners) are backlogged during a testsuite run scanning executables which are being compiled. When tmp.txt is closed, it gets added to Windows Defender's scan queue, but it can take long enough that it hasn't actually closed the file by the time fdstatus_main.ml tries to delete it.
The solution here is instead to open tmp.txt with O_SHARE_DELETE and then use an additional C stub to open the file with FILE_FLAG_DELETE_ON_CLOSE. This way, when fdstatus_main.ml terminates, the file is automatically by Windows "when it's ready". ocamltest merrily waits for the entire process group to terminate, as it usually does.
manual: alerts: document how to put alerts on all structure/signature items
Personally, I had no idea it was possible to put an alert on a let binding until I saw it mentioned on a github issue, because it's undocumented. Ideally, warning 53 would teach users that write:
let x = 1 [@@deprecated]
to write:
let x [@deprecated] = 1
instead, but I'm only changing doc here. And arguably, the first form could be interpreted as the second form, when the pattern contains a single variable (and fail otherwise), as that's how alerts works for all other binding constructs.
Commit
96622777af462b25d18af319b7041d31063c4abf
by gabriel.scherer
runtime: simplistic support for [ramp_up], [ramp_down]
During [ramp_up], the deallocation work coming from allocations is "suspended". It can be "resumed" by calling [ramp_down].
[ramp_up] does not currently count the total number of suspended allocations (this needs more domain state that is not reset on each major state), so the user would not know which value to provided to [ramp_down]. This will be added next.
Commit
4904473f55fcbaac0779060b8975e6dcd5c9c9a0
by clement.blaudeau
Removed abstraction of whole module types when avoiding a field
* Not backward compatible: code could use this behavior to create signatures with abstract module type fields, and place them in negative positions with [module type of], but it is highly unlikely intentional. There is no loss of expressivity: the desired resulting signature can still be obtained with an explicit signature constraint.
Commit
33b6307cf78ad1680e9e66d8bf6fd086b73add98
by gabriel.scherer
caml_update_dummy: the [Infix_tag] case is dead code
We stopped using dummies for values at a function type in OCaml 5.2, and the matching [caml_alloc_dummy_infix] function was removed in cd48dc0ac20f58fa460b55fafb75784d5a9d62f3.
Commit
ca72d32976636342855eec063cf391e1ef5ebc2c
by timmcgil
Check int and signal argument ranges for conversion functions
Signal_to_string and signal_to_int call invalid_arg if presented with an unrecognised negative number. signal_of_int calls invalid_arg for all negative numbers.
Commit
02a3282c112a5ed8eef7f4b6cd3bcbdb7366b41c
by antonin
Teach ocamltest about colors
As the compiler drivers and various ocaml tools, ocamltest now accepts the -color flag, and recognizes the OCAML_COLOR and NO_COLOR environment variables.
It uses them to pass a corresponding --color flag to diff, if it was detected during configure that diff supports this flag.
Commit
4b905fd93d349f41c416f9a06bd2433425fc67b6
by noreply
Fix hygiene check for generating `configure` (#13935)
* Allow check-configure.sh to be run locally
* Re-bash the autoconf test
* Simplify - and correct - check-configure.sh
The script contained an unused feature to be able to check PRs as one - this is never wanted, as we should always be checking that every commit has a valid committed configure script.
The mechanism for checking whether commits should be tested is shifted back to the actual trawl of the log. This is both simpler (because it can be done with a simple file list) but also fixes a bug, since commits which _didn't_ update configure could reset a failure from an earlier commit.
Commit
4c6856f0d3b74d1f98a053fd60ebf2d8d1c23007
by david.allsopp
Propagate file_descr cloexec correctly to channels
On Windows, the Unix library performs most operations on files using the Windows API directly, but the Unix.file_descr structure has the option for using an underlying CRT fd. In particular, these are needed for the {in,out}_channel_of_descr functions.
For some reason, the underlying _open_osfhandle call doesn't check for inheritability of the OS handle itself (it's slightly odd, because it does check other properties of it). By doing this ourselves, the resulting CRT fd now correctly propagates O_CLOEXEC.
Commit
84474967c692e5fb37e697ed9257fdc6495691d9
by david.allsopp
Set FLEXDIR when bootstrapping flexlink
Previously, the flexdll support objects were placed in both byte/bin and opt/bin with the copy of flexlink.exe when flexlink was being bootstrapped with OCaml. The objects are small, so the copying was not particulary onerous.
However, if opt/bin/flexlink.exe is a native Windows symlink (pointing to ../../flexlink.opt.exe) then Sys.executable_name when flexlink runs will point to the wrong place. While flexlink ought to be checking Sys.argv.(0) rather than Sys.executable_name, a better hardening is to be explicit and set the FLEXDIR environment variable to point to the directory containing the support objects. This also allows byte/bin/flexlink.exe and opt/bin/flexlink.exe to share the same copy of the objects.
Commit
4ee429d49dc03663eb6a91a8dd3f7da7ffaf7149
by clement.blaudeau
Refactor [is_functor_arg] table of env into [not_aliasable]
- In [env.ml], renamed the internal table used to track non aliasable modules from [Env_functor_arg] to [Env_not_aliasable], renamed the associated test function from [is_functor_arg] to [is_aliasable], renamed the [~arg] flags of some functions into [~noalias]
- In [includemod.ml], removed the redundant [can_alias] function
- In [typemod.ml], changed the error message for [Cannot_alias]
Commit
bd08fe5ff3125b0f0ef3549649420b3b4352d347
by david.allsopp
Add optional static and mimimal jobs
Tests a full `--disable-shared` build on Linux and also a Linux build with as many options disabled as possible (as the minimal build in the other-configs job on Jenkins also does).
The matrix is expanded by adding the 'CI: Full matrix' label to a pull request.
Commit
1183b2e68ac431be0717daa8d2858ce0834a936c
by david.allsopp
Fix test_create_cursor_failures.ml on Cygwin
If Cygwin is running "elevated" - which it is in CI - then it acts as though it's running as root. It intentionally activates SeBackupPrivilege, which thwarts the test_create_cursor_failures.ml test.
The OCaml testsuite will never require root privileges for anything meaningful, so ocamltest on Cygwin simply drops the SeBackupPrivilege when running on Cygwin, which means the test correctly fails.
Commit
c86afcccb3914ab718d88f8a89c169d508dd2da7
by clement.blaudeau
[refactor merge] Changed the return logic
- Typedtrees are no longer built inside [merge_constraint] but inside [transl_with], which removes the need for a special approximation case: merging always returns a module type, not a Typedtree. Changed [transl_with] to build the Typedtree there.
- Removed the [real_ids] mechanism that was used to store (imperatively) the list of affected paths. Now the patches (defined by [return], [return_payload] and [return_paths]) store both the resulting path and the list of affected paths.
- Added a [payload] mechanism used only for the type constraint case, where the replacement declaration is returned as an additional payload. Other cases return [None].
- Create separate functions ([merge_type], [merge_module], [merge_modtype])
- Extracted the post processing (wellformedness checks and substitutions) into an helper function [post_process]
Commit
69fa2d8ce47c79aa903be4ecac1dc1784eb535dc
by clement.blaudeau
[refactor merge] Extracted the recursive functions for deep constr
- Broke down the main merging function into three parts: [merge_signature], [patch_deep_item] and [patch_all]. The first two are mutually recursive and use an extra argument [~patch]. For now, the only patch provided (in [merge_type], [merge_module], etc) is [patch_all].
Commit
aeef23033f20749f3f422b7727c7d2214872e169
by clement.blaudeau
[refactor merge] Extracted the module case
- Moved the patching logic from [patch_all] to a specialized patch function in [merge_module]. Merged the common parts of the destructive and non-destructive cases
Commit
1c6248081df5f7b6954115f22ce17bc456a5d6f5
by gabriel.scherer
[breaking] Change the prototype of [caml_atomic_cas_field].
This is a breaking change because this function was (unfortunately) exposed outside CAML_INTERNALS, and is used by exactly one external user, you guessed it: https://github.com/ocaml-multicore/multicore-magic/blob/360c2e829c9addeca9ccaee1c71f4ad36bb14a79/src/Multicore_magic.mli#L181-L185 https://github.com/ocaml-multicore/multicore-magic/blob/360c2e829c9addeca9ccaee1c71f4ad36bb14a79/src/unboxed5/multicore_magic_atomic_array.ml#L36-L43
We chose to change the prototype to remain consistent with the naming convention for the new caml_atomic_*_field primitives, which will be added to support atomic record fields.
User code can easily adapt to this new prototype we are using, but not in a way that is compatible with both old and new versions of OCaml (not without some preprocessing at least).
Another option would be to expose
int caml_atomic_cas_field(value obj, intnat fld, value, value) value caml_atomic_cas_field_boxed(value obj, value vfld, value, value)
but no other group of primitives in the runtime uses this _boxed terminology, they instead use
int caml_atomic_cas_field_unboxed(value obj, intnat fld, value, value) value caml_atomic_cas_field(value obj, value vfld, value, value)
and this would again break compatiblity -- it is not easier to convert code to that two-version proposal, and not noticeably more efficient.
So in this case we decided to break compatibility (of an obscure, experimental, undocumented but exposed feature) in favor of consistency and simplificity of the result.
Commit
2122b66dba9cb7525ab68db7cbf30b09a4f97681
by gabriel.scherer
lambda: Add support for new atomic primitives.
Uses of existing atomic primitives %atomic_foo, which act on single-field references, are now translated into %atomic_foo_field, which act on a pointer and an offset -- passed as separate arguments.
In particular, note that the arity of the internal Lambda primitive Patomic_load increases by one with this patchset. (Initially we renamed it into Patomic_load_field but this creates a lot of churn for no clear benefits.)
We also support primitives of the form %atomic_foo_loc, which expects a pair of a pointer and an offset (as a single argument), as we proposed in the RFC on atomic fields https://github.com/ocaml/RFCs/pull/39 (but there is no language-level support for atomic record fields yet)
Commit
b1dea05488aa0a6a57d1a28834355e61492eaf39
by gabriel.scherer
Bootstrap.
To reproduce (see BOOTSTRAP.adoc for details): - go to the earlier commit "lambda: Add support for new atomic primitives" as a known-good state, build that one - keep the build artifacts around, come here and do `make bootstrap`
Commit
c139dade56f094a4e82bc1b22fa4982c0ce03acc
by gabriel.scherer
Bootstrap.
This bootstrap is not required by a compiler change, but it enables the use of the predefined type `'a atomic_loc` and the expression-former [%atomic.loc ...] in the standard library.
Commit
3e00aa54590360c0a97e471a259c5ad38ea12713
by noreply
Runtime events: dispatch the right event message type (#13970)
* Dispatch the right event message type
`type.runtime | type.user` does not really make sense as `.runtime` and `.user` are two projections of the same union type.
Prevents a MSVC 19.44.35109.1 warning:
runtime/runtime_events.c(595): warning C5287: operands are different enum types 'ev_runtime_message_type' and 'ev_user_message_type'; use an explicit cast to silence this warning
Co-authored-by: Gabriel Scherer <gabriel.scherer@gmail.com>
* Fit in 80 cols
---------
Co-authored-by: Gabriel Scherer <gabriel.scherer@gmail.com>
Commit
000a5c57ee9da59bd81d3132c55fcf4931bd9f99
by antonin
Fix unterminated-string-initialization warning
yacc/reader.c:62:1: error: initializer-string for character array is too long, array size is 32 but initializer has size 33 (including the null terminating character); did you mean to use the 'nonstring' attribute? [-Werror,-Wunterminated-string-initialization] 62 | "\000\000\000\000\000\000\000\000\376\377\377\207\376\377\377\007\000\000\000\000\000\000\000\000\377\377\177\377\377\377\177\377"; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ yacc/reader.c:64:1: error: initializer-string for character array is too long, array size is 32 but initializer has size 33 (including the null terminating character); did you mean to use the 'nonstring' attribute? [-Werror,-Wunterminated-string-initialization] 64 | "\000\000\000\000\200\000\377\003\376\377\377\207\376\377\377\007\000\000\000\000\000\000\000\000\377\377\177\377\377\377\177\377"; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Commit
5f663c525a88c76e93a7da27f2dad1aaacb2108c
by david.allsopp
Add caml_sys_proc_self_exe primitive
caml_executable_name is always called in native startup and for all the non-default bytecode linking mechanisms. Bytecode startup now always calls caml_executable_name, and this value is stored along with exe_name.
caml_sys_proc_self_exe returns this stored value as a string option. It returns None if caml_executable_name is not implemented on a given platform.
Commit
842ae77deebcc506adebc529e325bdaecbdd149d
by david.allsopp
Add Sys.runtime_executable
In native mode, same as Sys.executable_name, in bytecode, the path to the interpreter executing Sys.executable_name, which may not be the same from the same file.
Commit
7da23dc454005c035ba9c4ec61ebbd3d4be03fed
by gabriel.scherer
Better error messages on invalid recursive module definitions
Partially addresses issue of ambiguous error messages when no safe module is defined in a recursive module chain. The error messages should list out the full path of the values that cause the module to be unsafe.
Commit
5cc9a523d906307f59d1f00c3b5b1fb87c5d5d92
by gabriel.scherer
Format: place hint white spaces after the break hint
When formatting with margin > 9,
"@[aaaa@ bbbb@;<โ 0>cccc@]"
the `a` and `b` blocks fits inside the margin, and thus this text ought to be formatted as
aaaa bbbb cccc
However, before this commit `Format` rendered this text as
aaaa bbbb cccc
because it attributed the size of the horizontal contents of the `@;<โ 0>` break hint to the pending break hint `@ `. This commit fixes this issue by attributing the size of the horizontal contents of a break hint to the break hints itself rather than any pending break hint.
Commit
c36da49338c42193da62803f5eb4dbe31f28bf91
by gabriel.scherer
Reimplement generational stack scanning a la OCaml 4
Uses spare bits in return addresses to mark already-scanned stack frames. Currently works on - POWER - RISC-V - ARM 64-bits in Top Bits Ignore mode (i.e. under Linux but not under macOS)
Commit
86e2926895af77e2bac57838cd442eb0e842530a
by gabriel.scherer
ARM64: explicitly ignore top bits in return addresses
Unless the hardware is in top-bits-ignore mode already.
As a consequence, generational stack scanning is supported on all ARM64 platforms, incl. Apple Silicon / macOS.
The overhead of the extra masking instruction before every `ret` instruction is low: 1% to 1.5% code size increase; run-time increase is lost in the noise.
Commit
517e7bd99e9b379e4523a29bb2eddbfd042498b8
by pierre.boutillier
Use BUILD_PATH_PREFIX_MAP to sanitize debug event paths, do not rewrite shebang
1. bytecomp/emitcode.ml
Sanitize the paths in debug events using BUILD_PATH_PREFIX_MAP. However if the mapping has no effect, then do nothing.
2. bytecomp/bytelink.ml Do not do BUILD_PATH_PREFIX_MAP mapping of the path supplied by the user with the `-use-runtime` option. This is used to fill in the shebang part of the executable, and an abstract path is unlikely to work there.
Commit
a1263d64ddfdf5c5e91003e22abc10e3044258be
by samuel
Accept native freestanding targets at configure time
Accept `*-none` and `*-elf*` triplets for all the architectures with a native backend to describe the corresponding freestanding targets; `none` and `elf*` are the most commonly-used last components in triplets for freestanding targets Set `system` to `none` and `os_type` to `None` in such cases
Commit
8bc40f9d8dea61824c2c7f4bf773b59ba118c8d7
by samuel
Allow `*-ocaml` as target triplets to build freestanding cross compilers
Allow `ocaml` to be used as the last component of the target triplet in case we are using a custom toolchain for a freestanding target. The target triplet is then temporarily rewritten to "<arch>-none" to compute the canonical target. This allows to use a `*-*-ocaml-` prefixes (`x86_64-solo5-ocaml-`, for instance) to create cross-compiler toolchains dedicated to specific freestanding targets.
Commit
48efc58d67242349af56ae91944e3052c6cbd26b
by noreply
Add missing `item-attribute` rule for `let-binding` in documentation of attributes (#14077)
This reflects the grammar in https://github.com/ocaml/ocaml/blob/8761443617f229d5fe683ed2570aa79c8d64348a/parsing/parser.mly#L2742-L2759 and without this rule, the documentation doesn't account for forms like
Commit
6b82c95cf44071e86c3cbba2757520d0c227f339
by david.allsopp
Generalise Compmisc.init_path
Previously, Compmisc.init_path initialised the load path using Config.standard_library, but this can now be altered via an optional ?standard_library argument. This is used internally when testing compiler installations in order to allow Ccomp.call_linker to be used.
Commit
d9068d1a24b33b4837cdb7ba3d62aeebe26c3bc8
by olivier
Fix flakiness of TSan tests using flushes and synchronization
Co-authored-by: Fabrice Buoro <fabbing@free.fr>
All logging output is moved to stderr, the same output where TSan dumps its race reports. This is to help understanding what happens if the output of this test ever changes.
Additionally, a second synchronizing barrier is added to some tests to remove flakiness.
The existing barrier ensured that
1. there was a data race, by delaying the synchronizing `Domain.join` until after both domains had accessed the shared mutable field; and 2. that these accesses always happened in the same order (write first or read first).
The role of the new barrier is to enforce always the same order between the TSan report and logging lines such as `"Leaving f"`. Not enforcing that order was the source of flakiness in these tests.
Commit
072a905c85fbfcc1c6141d420ee4b4c3214790a7
by n.oje.bar
Symmetrize caml_sys_rename and caml_unix_rename
Make the libunix implementation common for Windows and POSIX.
The Windows implementation was using a simple call to MoveFileEx, but rename_os aliases to caml_win32_rename, which is a more portable and POSIX-like reimplementation, with fixes from #12320 and before.
Commit
055b3b16dbf74798829a68c8feab4ade081d40bb
by n.oje.bar
Symmetrize caml_sys_system_command and caml_unix_system
- caml_unix_system on Windows would raise ENOENT if the command string wasn't C safe. Prefer raising EINVAL as caml_sys_system_command. - caml_sys_system_command did not call _flushall on Windows as caml_unix_system did.
Commit
53ba0c848b9e8baf50654055a5019ea463a2723a
by Florian Angeletti
toplevel: printers must have a generic type
This commit adds an explicit test for checking that the type scheme of a value printers does not contain any non-generic type variables before being used to print a value.
Commit
9893cdd32e41d8347ccfcbd7dd60436913a8709f
by antonin
Use time_t instead of MSVC __time64_t
The documentation reads:
> `time` is a wrapper for `_time64` and `time_t` is, by default, > equivalent to `__time64_t`. If you need to force the compiler to > interpret `time_t` as the old 32-bit `time_t`, you can define > `_USE_32BIT_TIME_T`.
Commit
fc3294370566b4a0f749addd6b9be7cd2b19a7e0
by david.allsopp
Reduce tmpheader.exe to 4-5KiB on Windows
stdlib/headernt.c was adapted in OCaml 3.00 to reduce its size by avoiding the use of the CRT and using Windows API functions directly (this is a well-studied trick on Windows, principally as a puzzle for producing tiny binaries).
This got "regressed" slightly in OCaml 4.06, in the complex introduction of wide character support for Windows, as the mingw-w64 incantation required was unclear, so the entry point was changed to wmain, and the size of the header increased.
By switching from wcslen (a CRT function) to lstrlen (a Win32 API function), headernt.c again only requires kernel32.dll.
Additional flags are added for both ld (mingw-w64) and link (MSVC) to squeeze every last byte out of tmpheader.exe. The MSVC version of the header is once again no longer passed through strip, as this was found to be corrupting the executable (and had never been reducing its size anyway).
Commit
558c630b234bf6c9da5180cdda6945b91ff6c937
by david.allsopp
Remove unnecessary guards around unistd.h
Already updated to remove the actual test in s.h, since XPG1 (1985) required it and it is therefore part of the Single Unix Specification (1992), but the _WIN32 guard and the loading of s.h are unnecessary.
Commit
4a034852ad619b353f2fade2822f875e583545b6
by clement.blaudeau
Add the approx flag to merging of module constraints
- While module types constraints were using the [approx] flag to disable equivalence checking when merging, module constraints were only checked for cyclicity. Now, both use the same logic: the constraints is approximated and then merged in approx mode, where no equivalence check is done. It computes a better skeleton for the approximated signature, as the destructive substitutions are correctly removing the fields.
- Add tests borrowed (and adapted) from https://github.com/oxcaml/oxcaml/pull/4121
- This commit changes the [lookup_module_path ~load:false] for [lookup_module], where the load flag is not set to false. I should not have much impact for well-typed programs, as the loading would happen after the approximation phase anyway.
Commit
971d0b59c732bd6778b79f10b3f62e5e0ebe4d67
by clement.blaudeau
Add a [merge_type_approx] for approximation of type constraints
This commit introduces a new function to specifically deal with merging type constraints in approximation mode: - destructive constraints actually remove the constrained field, to prevent incorrect approximation (specifically, incorrect shadowing) - non-destructive constraints are treated as an identity patch, where the constrained field is replaced by itself. This allows us to reuse the normal merging infrastructure and fail early in case of ill-formed constraints where the field is not present (other forms of ill-formedness are caught later)
The [post_process] function is made aware of the approximation flag to disable wellformedness checks
Commit
f9025a6d8644b69ccef173847a3d662210a023c4
by clement.blaudeau
Refactor post_process of signature merging
The [post_process] function for signature merging was taking two linked arguments: a [~destructive] flag and a [replace] function to apply only if the flag was set to [true]. This commit combines the two into a single optional function [replace] and clarifies some documentation comments.
Commit
fa858d9d631782d95d7d5a9c5abe595e8ea506a6
by Florian Angeletti
typechecker: fix an internal error due to wrong exception
`Ctype.mcomp` was raising an errortrace when two types were incompatible in two rare cases, whereas the function was specified to raise the `Incompatible` exception. This was fine for internal uses of `mcomp` within `Ctype`, because call to `mcomp` went through `mcomp_for` that transformed the `Incompatible` exception into an errortrace. However, this leads to internal errors for others use of `Ctype.mcomp` that were only expecting an `Incompatible` exception and not an errortrace.
This simple fix replaces the two raises of errortrace by a raise of `Incompatible`.
Commit
69ed94cf28d9485d47bb572bacfa545b6959a9df
by gabriel.scherer
minor gc: more consistent memprof logging
In https://github.com/ocaml/ocaml/pull/13580#issuecomment-3092253963 jmid reports that he needed to tweak the GC verbosity setting to avoid getting spammed by minor-gc messages when debugging an assertion failure.
The other sub-phases of the GC minor all uses `caml_gc_log` rather than CAML_GC_MESSAGE, and do not seem to cause similar spamming issues. Fixing the code to be consistent will avoid inconsistent verbosity levels in end-user scripts.
Commit
8f0d4f4b77215029fcc3f331cfc0cee3cdff051b
by gabriel.scherer
runtime: free the minor heap when leaving STW participants
The reserved address space for the minor heap area is a global resource shared by all domains; each domain owns a portion of it, within which it commits a part for its minor heap. (Having contiguous address space allows for an efficient Is_young check.). When we need more reserved space because the user increased the minor heap size, we use a STW section to change the reservation: each domain in the STW section first decommits its minor heap, a single domain changes the reserved area, and then each domain re-commits its minor heap.
If a domain does not participate to STW sections, the boundaries of its minor heap will change without the domain decommitting the previous minor heap first. If the same domain structure is used for a newly spawned domain later on, it will start by decommitting its minor heap following the new boundaries, which is incorrect as it never committed this adress range in the first place.
(In practice calling `caml_mem_decommit` incorrectly in this way does not appear to crash the program. I think this is because `decommit` has a fairly liberal behavior, it will happily do nothing if the memory range is not committed. The code remains logically wrong, and could become a hard failure if other parts of the runtime change in reasonable ways later on.)
The present commit ensures that we systematically decommit the minor heap of each domain when it leaves the set of STW participants. This way, only STW-active domains have their minor heap allocated, and changing the minor heap address space within STW section works as intended.
(I tried to remove the new call to `free_minor_heap` in `domain_terminate`, and checked that the testsuite fails in debug mode when the `allocate_minor_heap` call in `domain_create` later on notices an already-committed minor heap.)
Commit
e93ba211dbd022139cfc36e05e3d4e0185925fcd
by Florian Angeletti
Take fixed instances of polys where appropriate
This fixes a bug in the interaction between polymorphic variants and polymorphic parameters. The actual bug fix is just changing two falses to trues but I also changed the instance_poly interface to make similar mistakes less likely to happen.
Commit
02746081cc1310daafc67122365e514032de2a54
by david.allsopp
Fix dependency generation when tools are disabled
The OPTIONAL_BYTECODE_TOOLS, OPTIONAL_NATIVE_TOOLS and OPTIONAL_LIBRARIES should be being used to affect build and installation, not definitions. If ocamltest et al were disabled, then the definitions of these programs were omitted, which prevents the reproducible generation of dependency information.
Commit
f627d6356224bd93505964200cd39a0f57451416
by david.allsopp
Fix generation of config.status for dynlink
Running config.status works correctly, but individually requesting links in otherlibs/dynlink did not because the names were specified using a shell variable (i.e. at configure-time) instead of a m4sh variable (i.e. at autoconf-time).
Commit
08d94e782ed2215aad075e011afd3efa02585524
by gabriel.scherer
Minor heap reservation: change the terminology
The current codebase use 'caml_minor_heaps_{start,end}' for the boundaries of a global address space that is reserved, 'dom->caml_minor_heap_area_{start,end}' for a 'minor heap area', a segment of this address space that is owned by each domain, and then finally 'dom->young_{start,end}' for the prefix of this segment that is actually committed and used as the minor heap of each domain. Some comments refer to the latter as the 'minor heap arena', following terminology from the Retrofitting Parallelism into OCaml paper.
On a suggestion by KC, I am trying to make the naming scheme more regular by consistently using 'reservation' for a reserved block of address space:
- Use 'minor heaps reservation' for the global reservation. Its boundaries remain stored in 'caml_minor_heaps_{start,end}' to avoid compatibility issues in third-party code.
- Use 'minor heap reservation' for the per-domain segment of the global reservation. Its boundaries are stored in 'dom->minor_heap_reservation_{start,end}'.
- Use 'minor heap' for the prefix of the minor heap reservation that is actually committed, whose boundaries remain 'dom->young_{start,end}'.
My PR #14158 merged today introduced a bug in the logic to resize the minor heaps reservation. It added the following to the `free_minor_heap_arena` function:
domain_state->minor_heap_wsz = 0;
Doing this is correct when we are freeing the minor heap arena of a domain that is leaving the STW participant set (the focus of #14158); it is also correct in
int caml_reallocate_minor_heap_arena(asize_t wsize) { free_minor_heap_arena(); return allocate_minor_heap_arena(wsize); }
which is called to change the size of the memory area, so zeroing it in `free` before setting it in `allocate` is fine. However, it is *not* correct in
if (allocate_minor_heap_arena(Caml_state->minor_heap_wsz) < 0) { caml_fatal_error("Fatal error: No memory for minor heap arena"); } }
This function changes the global minor heaps reservation during a STW event where each domain first deallocates its arena and then reallocates it in the new reservation. The problem is that `free_minor_heap_arena` now changes the value of `Caml_state->minor_heap_wsz` to 0, so the re-allocation that follows will try to allocate a 0-word (in fact a 512-word due to the page-alignment normalization logic) arena.
This bug can only be encountered by calling `caml_update_minor_heap_max`, so it affects few programs.
I see two approaches to fix it:
1. we could remove the zeroing of `minor_heap_wsz`, and instead use the previous check `young_start == NULL && young_end == NULL` to detect uninitialized arenas
2. ... or we do assume that `free_minor_heap_arena` will unset the arena size (which is reasonable), and we preserve the desired size value within the `stw_resize_minor_heaps_reservation` function.
The present commit implements approach (2). I prefer to avoid a situation (as with (1)) where the `free` would leave the state only partially initialized, and it would be important for correctness.
Commit
bab7aa8a5838830b5bb41a3c4b12dcd058665344
by gabriel.scherer
Memprof.start replaces any existing profile in the domain, Memprof.is_sampling
The change to Memprof.start increases its compositionality while conforming to the previous behaviour (it simply fails in fewer situations). This is necessary for us to implement the Memprof interface on top of the package memprof-limits.
The new function is_sampling is for clients that do want to fail early, e.g. when detecting that two Memprof clients are interfering.
Commit
34a810a42b7aa75423568a14c00a72b0a78fddb8
by noreply
runtime: host aligned fibers inside the fiber cache whenever possible (#14169)
* When growing a fiber, zero the alignment word before computing the next size in order to make this new size fits inside the fiber cache. * Add an assertion to check that small fibers are using the cache.
Commit
4f2163db2e1f74134ea67421538bc211b30cb713
by timmcgil
Add runtime counters EV_C_MINOR_PROMOTED_WORDS and EV_C_MINOR_ALLOCATED_WORDS.
EV_C_MINOR_PROMOTED_WORDS reports words promoted by minor GC and EV_C_MINOR_ALLOCATED_WORDS reports words allocated by minor GC. Both have equivalent bytes counters.
Update the documentation for EV_C_MINOR_PROMOTED and EV_C_MINOR_ALLOCATED to qualify scope of the values reported.
Commit
8b1e26db2d9067c62680a0607e9f2f1255f93bde
by Florian Angeletti
functor error messages: don't forget equality
Higher-level error messages for functors recompute inclusion checks when trying to discover more macro-level error messages. For this reconstruction to be accurate, those computations must use the same environment than the one used when detecting the original problem.
In particular, this environment must include equalities added during the pairing of types and modules during the signature inclusion test. For instance, in
module M: sig type t module F(X:sig val f:t val g:int end): sig end end = struct type t module F(X:sig val f:t val g:float end)= struct end end
we must remember that the interface-side `t` is equal to the implementation-side `t`.
This part of the inclusion checking environment was ignored before this commit leading to non-sensical error messages complaining that `t` is not compatible with `t`.
This commit extends the captured environment for errors in signature to include the substitution recording the equalities between items on both side of the check.
Commit
8f1ba17f68f6f317f1ebfa9792c6f6793cbc92b9
by david.allsopp
Fix st_stubs.c compilation with mingw-w64 13.0.0
In mingw-w64 13.0.0, time.h now causes pthread_compat.h to be included which as a side-effect sets up the macros for declspec(dllimport). Since caml/osdeps.h uses time.h, this means that the macro changes designed to ensure the API functions are properly decorated doesn't get applied, and and RELOC_REL32 errors abound.
The fix for now is to ensure that the macros are setup to control pthread_compat.h at the very beginning of the file.
Commit
374aee416bc98d2e6db15b495f05bbb7d88ecfae
by noreply
Fix test-in-prefix (#14216)
Add `flexlink.exe` and `ocamllex.exe` to the list of files that do not link with the Config module. Normally, these are symlinks to `*.byte.exe` or `*.opt.exe` but (depending on Windows shenanigans) `configure` may decide to use `cp` instead of `ln -s`.
Commit
597255978e92279f168588dd793d9ebf83652ed1
by david.allsopp
Don't add a double-separator when locating ld.conf
When calculating the full path for ld.conf, the runtime unconditionally concatenated "/ld.conf". This is harmless when the separators appear in the middle of a path ("/usr/local/lib/ocaml//ld.conf" is equivalent to the version with only single slashes), but it is technically incorrect for two corners cases with OCAMLLIB and CAMLLIB:
- if either is explicitly set to "/" then "//ld.conf" is _not_ the same file as "/ld.conf". This is mildly relevant on Windows and Cygwin where the two initial slashes (including as "\/" for native Windows) will be interpreted as a UNC path - if either is explicitly blank, then "ld.conf" (i.e. ld.conf in the current directory) is a less illogical file to open than "/ld.conf"
Commit
bba973cf1a6c48ddbcd5b4b61a60d3b4a076f09a
by david.allsopp
Remove caml_get_stdlib_location
The function was only ever added to share the logic between dynlink.c and startup_byt.c - now that dynlink.c doesn't require it, move the function to startup_byt.c and make it internal again.
Commit
a64ab59378ec28d781cc8543e195953605bcd4fe
by david.allsopp
Use caml_parse_ld_conf in ocamlc
Eliminate the need for two implementations of the parsing logic for ld.conf by sharing the C implementation (which must exist, since it's part of bytecode startup) with the bytecode compiler, replacing Dll.ld_conf_contents
Commit
eba662975bb30f8cd1245401bc03f5f1a3ee053f
by david.allsopp
Fix the detection of Cygwin-like build environments
Both Cygwin and MSYS2 are now consistently detected on MSYS2. In particular, this means that ./configure --prefix $PWD/install and similar will cause the prefix to be correctly translated to a Windows path, as already happens on Cygwin.
Commit
ddcdbf92f49d8837d3ee213ff108d6953624cc98
by david.allsopp
Preserve backslashes in --prefix
Previously, the --prefix argument was always normalised with cygpath -m which meant that regardless of the argument, the paths used in the compiler would always use slashes.
This behaviour is preserved if a slash is detected in the argument, i.e. the caller explicitly uses mixed notation (e.g. `--prefix=C:/Prefix` or `--prefix $PWD/install`). In particular, it means that a Cygwin-style path will be correctly converted to a Windows-style path.
If the path uses backslashes, then it is still converted to use forward slashes for the installation commands, but the backslashes are otherwise preserved and used within the build itself.
Commit
fd4b73c7604dbceeb5b24f0cd5ed080c59089482
by david.allsopp
Harden startup of -custom executables
By default, ocamlrun first tries to resolve argv[0] to determine where the bytecode image is and then tries opening the executable image itself. This is obviously correct for ocamlrun, when being called using a shebang or executable header, but it's not correct for -custom executables where we _know_ that the bytecode image should be with the executable. To achieve this, a new mode is added to caml_byte_program_mode (and the existing ones renamed) such that caml_byte_program_mode is now STANDARD (for ocamlrun - the existing behaviour), APPENDED (for -custom executables - the new behaviour) and EMBEDDED (for -output-complete-exe/-output-obj - the original use of it).
The mode is also set directly by the linker, rather than having a default in libcamlrun which is then overridden by the startup code for -output-complete-exe.
In the new APPENDED mode, if caml_executable_name is implemented (i.e. it returns a string) then this file _must_ contain the bytecode image and no other mechanisms are used. On platforms where caml_executable_name is not implemented, APPENDED falls back to STANDARD for compatibility.
Technically, this stops an argv[0] injection attack on setuid/setgid -custom bytecode executables, although setuid should be used with -output-complete-exe, if at all.
Commit
12419deb0c08de9b4a72d80b1cdd674e6498c15b
by david.allsopp
Interpret . in runtime-launch-info
The runtime-launch-info file includes the location of the binary directory. The compiler is extended so that . refers to the directory of the compiler binary.
Commit
6b8f6fb9c10e5a9525d18d616b1f66ca10b6c816
by david.allsopp
Add caml_runtime_standard_library_default
Previously, the bytecode runtime just used OCAML_STDLIB_DIR from build_config.h. This value is now stored once in dynlink.o as caml_runtime_standard_library_default.
Commit
1009f7fd8b0bd93c166cac5d5ffb29824fc37bbd
by david.allsopp
Add %standard_library_default
%standard_library_default allows Config.standard_library_default to be converted to a compile-time derived value, as with existing compile-time constants such as %backend_type, etc. This paves the way for allowing Config.standard_library_default to be changed at link-time, rather than fixed when the Config module itself is compiled.
Commit
54e518173e8be26b212875d85b5b38b2d9004786
by david.allsopp
Use %standard_library_default in Config
Config.standard_library_default is now implemented using the %standard_library_default primitive. This allows a convenient test which can be added for `-set-runtime-default`.
The change also makes the host-like nature of of Config.standard_library_default clearer, as the build of the cross-compiler must now (correctly) specify the location of its (target) Standard Library.
Commit
415a2881b38ee0afafbed036130cd49870f22688
by david.allsopp
Allow libdir to be found relative to bindir
When configured with --with-relative-libdir, the runtime uses the directory of the executable to determine the location of the Standard Library. Thus, ocamlrun and the compilers look for ../lib/ocaml by default.
This is implemented by changing caml_standard_library_default to be a relative path, and then computing the actual value at startup (for bytecode) and when queried (for native).
Executables (and objects) produced by the compiler always have an absolute value of caml_standard_library_default. ocamlc.opt and ocamlopt.opt are built using -set-runtime-default to force caml_standard_library_default to be a relative value.
Commit
141563c2a29a98bd7e025d1029bb292d44413a36
by david.allsopp
Detect but ignore -fdebug-prefix-map on mingw-w64
mingw-w64 is based on GCC, so supports -fdebug-prefix-map, but the test for it is skipped in configure. The test is no longer skipped (which means that Config.c_has_debug_prefix_map returns true) but the flag is still explicitly not used by the compilers (as before).
Commit
a7f618a9ac8a010c0b60940d508e2cc1b31d2662
by david.allsopp
Add -launch-method to ocamlc
When linking a normal bytecode executable, allows an explicit selection of either the executable or shebang header, regardless of the value in runtime-launch-info.
Commit
8d721dee1ccb87e52dc29c6a17031c92acbda0ef
by david.allsopp
Remove metadata from runtime-launch-info
-launch-method encapsulates the first line of runtime-launch-info. The argument to -launch-method is extended slightly to encompass the second line, thus `-launch-method 'sh /usr/local/bin'` represents the default runtime-launch-info file on Unix. Additional fields are added to Config so that the installed compiler simply uses default values, rather than reading the two lines from runtime-launch-info. The build of the compiler itself explicitly uses `-launch-method`, which leaves only the executable launcher compiled from stdlib/header.c in runtime-launch-info.
Commit
3e80494def1180ecae5bd23422ca5b516638633b
by david.allsopp
Add -runtime-search to ocamlc
-runtime-search {disable|enable|always} adds new features to the launcher used for bytecode executables which do not embed their own runtime. By default, the header continues to behave as before - the launcher will attempt to start the runtime using the absolute path which the compiler was configured with.
The new search mode will then search for the runtime first in the directory containing the running executable and then in PATH.
Commit
416a4c50cb588894f70ca64858c38c864ef7b31e
by david.allsopp
Build suffixed shared runtimes
New names for libcamlrun_shared.so and libasmrun_shared.so without the _shared suffix and using the target triplet and runtime ID. Both ocamlc and ocamlopt explicitly recognise `-runtime-variant _shared` and select the correct name.
Symbolic links for libcamlrun_shared.so and libasmrun_shared.so to allow any C programs which linked against the the output of `-output-obj` to continue to work.
Commit
59f845c6d1af41979df8e4ceea856863b21a130b
by david.allsopp
Add runtime suffixes to bytecode stub libraries
ocamlc -dllib-suffixed appends the runtime's host triplet and bytecode runtime ID to the supplied name when searching for the DLL, and records the base name only in .cma / executable files.
ocamlmklib -suffixed instructs ocamlmklib to use -dllib-suffixed when generating .cma files instead of -dllib.
The effect is that stub libraries built this way have names which will be unique for a given configuration of OCaml and so will be ignored by other runtimes.
Commit
ed990f2ae3e844b364233eb03c0e4db9b5455168
by david.allsopp
Add --enable-runtime-search[-target] options
--enable-runtime-search controls the -runtime-search setting used to build the compiler's own bytecode executables; --enable-runtime-search-target controls the default value of -runtime-search that ocamlc itself uses.
Commit
df579803b7505620ee05eba251abe27c686ca4cd
by david.allsopp
Don't explicitly install main.cmx and optmain.cmx
It's necessary to install main.o and optmain.o because the two modules are not part of ocamlbytecomp.cmxa and ocamloptcomp.cmxa, but the .cmx files are already installed as part of wildcard patterns on driver/
Commit
c3a033fdefcf04a44cc084c9da132c7b92506df3
by david.allsopp
Use implicit names when installing ocamldoc
The ocamldoc binaries were installed using a relative path. They're the only binaries installed that way - switch them to use an implicit path, as it's easier to make an implicit path subsequently relative if needed than vice versa.
Commit
d821830c7810cc8623f09fef14930db3a9c1acef
by david.allsopp
Keep the stripped bytecode binaries
No need to remove things during install when it will be done later by a package manager. Also prepares for the possibility of letting something else install the binary subsequently.
Commit
9f7b7d0f58fb61d6686f775ee37c58dfe8a33427
by david.allsopp
Straighten out the INSTALL_* Makefile variables
Move various definitions around such that merged Makefile variables are all in Makefile.common. Sanitise the names to be closer to INSTALL_LIBDIR_subdirname.
Commit
31f5dba2cee0a2d8214db89060ade781175b13e6
by david.allsopp
Add SUBDIR_NAME to principal Makefiles
SUBDIR_NAME goes along with ROOTDIR - where ROOTDIR is the relative path from Makefile back to the root directory, SUBDIR_NAME is the implicit path from the root directory to Makefile.
Commit
fb71f2e967d1549b5e8c05907172e7cf9dc99bf7
by david.allsopp
Use macros to generate installation commands
make install works as it did before. All the commands in the install targets now go through a macro call which allows the semantic intent of each command to be more clearly specified.
Commit
d613b3e2af7fd2cc6b01a5645f525f9f0ea92339
by david.allsopp
Add additional modes to make install
make [INSTALL_MODE=install] install - installs the compiler, as normal make INSTALL_MODE=display install - displays the operations needed for make INSTALL_MODE=list install - lists the files and symbolic links which are installed
Commit
5b616b037a01ca1549fc013c444eafff8c0d6271
by david.allsopp
Generate an opam .install file and support script
make INSTALL_MODE=opam install generates $OPAM_PACKAGE_NAME.install and $OPAM_PACKAGE_NAME-fixup.sh ($OPAM_PACKAGE_NAME defaults to ocaml-compiler). Nothing is installed by this mode. The fixup.sh script is intentionally not made executable (it should be invoked explicitly with sh) and creates symbolic links, if required, and also manually copies the files to the doc dir, as the .install file format doesn't allow the correct location to be specified.
Commit
8f9c476ce7eec69583b43fbdc88c61d60e3e952c
by david.allsopp
Mark input dependencies as build
Mitigates an issue with opam install --assume-built since build dependencies are ignored. The semantics should remain consistent: in particular, as there is only a single version of each of these packages, the key issue is that removing the package will still trigger the correct behaviour as the dependency graph will change.
Commit
114dfa468aaa3c37296ab599ce6634a2443b847e
by david.allsopp
Generator script for ocaml.config in opam
This script is installed in opam by the ocaml-config package and then used by its ocaml virtual packages to probe the compiler installation and record various pieces of information about it by generating ocaml.config
Commit
dc5542ac4be9aa0187fbed32f7ddf085372d54e8
by david.allsopp
Windows fixes to ocaml-config
- Wrong separator used for generating CAML_LD_LIBRARY_PATH on Windows - Fixed detection for the native-tools variable on Windows - Support dkml-base-compiler
Commit
c1198562f419b3935a5c33abf511d72769fbac09
by david.allsopp
Fully unify and update gen_ocaml_config.ml
There was technically a slightly tailored version of gen_ocaml_config.ml sitting in the history to support the OCaml 3.07 package (which is the oldest compiler archived in opam-repository).
The ocaml-config package originally stored this script in the files/ subdirectory, but policy changes in 2025 mean it was being referenced from an archive repository. This script can now be referenced directly by opam's ocaml package, removing the need for an additional package in the dependency cone and also meaning that the script no longer has to be installed in the switch.
The substs mechanism has been replaced in favour of passing the required arguments on the command line. The previous mechanism was an artefact of the way this script was previously generated, rather than necessarily a clear way of writing it. It means this script is now simply an OCaml script and has the added benefit of allowing each ocaml package to specify the appropriate additional version components (e.g. spacetime, nnp, etc.) rather than having each script confusingly containing all of them.
Commit
eab002fce0c9cb1ed1bd3b7bba5d3cfc2d36ed81
by david.allsopp
Remove unnecessary Cygwin path workarounds
This is no longer required (nor does it work). Cygwin 1.5.20 (July 2006) added the transparent_exe option to the CYGWIN environment variable which made open behave in the same way as stat. Cygwin 1.7.1 (December 2009 and, despite the version number, the first release of Cygwin 1.7) made this behaviour default (and removed the ability to turn it off).
Commit
2fe328bcf21000e4c0a31402945bea70ba073777
by david.allsopp
Simplify the interface of caml_attempt_open
Previously, caml_attempt_open received a pointer to the string and passed this string to caml_search_exe_in_path. This function allocates a fresh string which was then assigned to that pointer.
This interface was a little cryptic to follow - in particular, there were several scenarios in which the result was not being freed. It's also not entirely clear that caml_attempt_open involved a PATH-search, which appears to have been applied in error when CAML_DEBUG_FILE was added.
Commit
1d2e46c1bf9776b5c077ee76aafafea3492466c6
by david.allsopp
Preserve argv[0] on Unix in the executable header
Makes the behaviour of the Unix and Windows versions of the header equivalent, in particular it means that if argv[0] doesn't describe the executable, more things fail for Cygwin.
Commit
86caf8eeca595d658fcd93cc9c31bfad499da844
by david.allsopp
Fix STARTUPINFO structure in stdlib/header.c
If a CRT application (including, therefore, another OCaml program), exec's a bytecode program which uses the executable header, the cbReserved2 and lpReserved2 fields of the STARTUPINFO structure are quietly used by the CRT to pass handle information about open fds.
The status checker in the cloexec.ml test has C stubs, which causes ocamltest to compile the bytecode version with -custom. The test is split to have an additional intermediate pure OCaml program which simply repeats either the Unix.execv or Unix.create_process call with the Sys.argv it was applied. In bytecode, ocamltest compiles this program just with -use-runtime, which means it uses the executable header on Windows.
Without the change to header.c in this commit, that test begins to fail in bytecode on Windows, because the CRT information about inherited handles is not passed on to the status checker (the HANDLE values will have been inherited, as that's a kernel function, but the CRT structures regarding the fds are not initialised, which is what the checker than looks at).
The fix here simplifies the code considerably - rather than initialising a fresh STARTUPINFO structure, we simply use GetStartupInfo to retrieve the one which was used to create the process itself and pass that to CreateProcess - cbReserved2 and lpReserved2 are therefore passed on, and the cloexec.ml test passes again.
Commit
e7ffd7c05d1a92a651a654e7a8e7a9ad9cc83b15
by david.allsopp
Share image fd between header.c and startup_byt.c
The bytecode executable launcher (stdlib/header.c) can _only_ invoke ocamlrun after it has opened itself in order to find out the name of the runtime to execute. However, once ocamlrun is exec'd, the knowledge of this file was previously lost and if it could not be recovered from argv[0], then execution fails.
This new approach, for both Windows and Unix, instead keeps the fd for the bytecode image open and passes its number to ocamlrun as __OCAML_EXEC_FD in the environment. ocamlrun detects this environment variable and uses that fd to load the bytecode image. If this fails, the runtime does _not_ fallback to any other mechanisms.
On Windows, it is possible to recover the filename from a HANDLE. It is not portably possible to do this on Unix, so the filename which was opened by the stdlib/header.c is instead appended to the environment variable.
Commit
b76df17b33ec076ea32629dc744b9f4c9b8c3ec5
by david.allsopp
Add comments on the handling of blank env vars
Clarify in various places the behaviour of reading an environment variable which is "Set But Null" (i.e. equal to the empty string as opposed to unset)
Commit
2c40b1b129389d0b3aa08fe9edbd02460ea35f99
by david.allsopp
Propagate non-empty tmpdir in ocamlyacc
By making tmpdir explicitly "." in the rare corner-case of TMPDIR being set to the empty string on Windows, it becomes unnecessary to keep checking whether tmpdir is zero-length.
Commit
6444211b4f735c77aa9a1f24009187477ac4a018
by david.allsopp
Use non-raising Sys.getenv_opt in win32unix
Use the non-raising Sys.getenv_opt internally in the Windows implementation of the Unix module. In passing, treat the highly unlikely corner case of COMSPEC being "Set But Null" as if COMSPEC were not set.
Commit
5ea225bc30136076fa4a9f8ea01f3f73c2301fa7
by david.allsopp
Ignore empty strings in Compmisc.set_from_env
Previously, invocations such as
OCAML_COLOR= ocamlopt
emitted warning 46 (bad-env-variable) which is not particularly helpful. At present, Compmisc.set_from_env is used to implement OCAML_COLOR and OCAML_ERROR_STYLE, neither of which attached significance to the empty string. Compmisc.set_from_env is there changed to explictly ignore the environment variable value if it is "".
Commit
db7f15f9f11b73bd721d3d1056e8abed9dec2362
by david.allsopp
Ignore empty environment variables in ocamltest
Most uses in ocamltest were via its safe_getenv function which already made unset and empty equivalent. This generalises the underlying getenv_with_default_value function always to return the default value if the variable is _either_ unset or set to the empty string.
Commit
aea17690ac74a44770d634809d16fd5887bbef80
by david.allsopp
Treat OCAML_BINANNOT_WITHENV as unset when empty
This environment is intended to be used in the context of OCAML_BINANNOT_WITHENV=1 to enable it, so treat OCAML_BINANNOT_WITHENV= as if it weren't set at all.
Commit
a69fbd2890de42657fb70a05c5c50212877d10f3
by david.allsopp
Ignore empty OCAML_RUNTIME_EVENTS_ variables
The handling of OCAML_RUNTIME_EVENTS_DIR was incorrect if the value as "Set But Null" and the intention in the manual for OCAML_RUNTIME_EVENTS_START and OCAML_RUNTIME_EVENTS_PRESERVE is clearly that they should be set to non-empty values in order to trigger the required effect.
All three variables are now ignored if they are set, but to an empty string.
Commit
1f6d830848d24e77a3529284acb13e37858a602d
by david.allsopp
Harden processing of SOURCE_DATE_EPOCH in ocamldoc
Previously, running:
SOURCE_DATE_EPOCH= ocamldoc
resulted in an uncaught Failure "float_of_string" exception. The processing of SOURCE_DATE_EPOCH is firstly hardened to cope with parsing errors and then a one-time warning is displayed the first time it's actually used (at present it's only required in Odoc_man).
Commit
01c197f549775cb88b4353423f6949cf41a22abc
by david.allsopp
Ignore blank OCAMLRUNPARAM, OCAMLLIB and CAMLLIB
Previously, if OCAMLRUNPARAM was "Set But Null", CAMLRUNPARAM was completely ignored. Similarly, a "Set But Null" value for OCAMLLIB caused CAMLLIB and subsequently the default location of the Standard Library to be ignored. Especially for OCAMLLIB/CAMLLIB, this behaviour is counter-intuitive, as it's highly unlikely to be useful.
Now, if OCAMLLIB is set, but to the empty string, then CAMLLIB is checked and, more importantly, if it is either not set or also set to the empty string then the default location of the Standard Library is still used. For consistency, if OCAMLRUNPARAM is set, but to the empty string, then CAMLRUNPARAM is read.
Commit
3f37ec5d9cd7792a47d3c92b8e4eba5e6a8fb302
by david.allsopp
Ignore empty components in PATH-like things
POSIX recognises empty components in a PATH-like variable as meaning "." (the current directory). This is reflected in the processing of OCAMLTOP_INCLUDE_PATH, CAML_LD_LIBRARY_PATH and ld.conf where either a blank component or a blank line is interpreted as "."
Somewhat confusingly, this processing is applied inconsistently between Unix and Windows (it's confusing given that Windows more readily includes the current working directory by default in PATH searches).
It also has the side-effect that a "Set But Null" environment variable is interpreted as "." which counter-intuitively makes CAML_LD_LIBRARY_PATH= ocamlrun add the current working directory to the search path.
Blank lines and empty components of both OCAMLTOP_INCLUDE_PATH and CAML_LD_LIBRARY_PATH are now ignored. The current working directory can still be explicitly included, of course, by adding a "." entry/line where required.
Commit
d0e76ff91aefb76422a1d88f92b971a8ff36ee1a
by david.allsopp
Improve the "dynamic loading not supported" error
It is possible, especially when using Dynlink, to end up in the situation where a bytecode runtime which doesn't support dynamic loading is asked to load support DLLs (e.g. a bytecode image with a DLLS section, or a cma archive passed to the toplevel/Dynlink which has a non-empty lib_dllibs list).
Previously, the error message would refer to the name of the first DLL being loaded and simply state that dynamic loading is not supported. The confusing part is that typically this would refer to a DLL which is not on the system.
Now, the bytecode linker only writes DLLS and DLPT when there are entries to write in them, and the runtime, toplevel and Dynlink provide a direct explanation that dynamic loading is needed, but is not available. In particular, the error now refers to the file which is being loaded (i.e. the bytecode executable or the .cma file) rather than a .so file which doesn't exist.
Commit
2eda1aec16d0dc7379420a18f82c9f7110eb1d6c
by david.allsopp
Free tables after handing off to Dynlink
In the debug runtime, caml_prim_name_table remains for the lifetime of the program, as it's used by instrtrace.c, but in normal operation, once the list of primitives has been handed over to Dynlink, it's no longer required. In the normal runtime, it's now freed after this handover.
In passing, strings themselves are no longer duplicated, as the code path is a good deal simpler than it used to be, and the bytecode section itself can reliably be used as the underlying buffer for caml_prim_name_table.
caml_shared_libs_path is only kept at all to be handed over to Dynlink. It, along with the two underlying buffers for CAML_LD_LIBRARY_PATH and ld.conf are freed after the call.
Commit
8cb96287d83bff483219a007e9eafc09b4dc5139
by david.allsopp
Preserve option R in caml_runtime_parameters
The setting for R was previous omitted in Sys.runtime_parameters, since it was only processed directly by the Hashtbl module and not stored in the runtime. Option R is now processed in caml_parase_ocamlrunparam and stored to be accessed and updated via new primitives for the Hashtbl module.
Co-authored-by: David Allsopp <david.allsopp@metastack.com>
Commit
57412ce0a9cf977e5e92059eabb7a9aecc22cbed
by david.allsopp
Add -set-runtime-default to the compilers
When linking an executable, allows to set default to OCAMLRUNPARAM values. This new OCAMLRUNPARAM string is accessible using the "caml_executable_ocamlrunparam" symbol or is embedded in a bytecode section.
Co-authored-by: David Allsopp <david.allsopp@metastack.com>
Commit
064aa0f456a2f269c3e6b1397440031df67f7cfb
by david.allsopp
Test -set-runtime-default in test-in-prefix
The test programs in the run after the prefix has been renamed are compiled with `-set-runtime-default R`, and the test program verifies that Hashtbl.is_randomized returns the expected value.
Commit
a5a53c9aec61969c02aeef4748e3a7293c1840ff
by david.allsopp
Support c=1 mode in ocamlrun
The runtime's pooling mode has a slight Catch-22 problem for ocamlrun when enabled using -set-runtime-default. Opening the bytecode file and reading the ORUN section requires the memory subsystem.
In this revised version, caml_main in bytecode is particularly careful to track exactly what will have been allocated prior to reading the ORUN section and if ORUN requires the system to start pooling mode, the runtime now takes temporary malloc'd copies of everything which has been made so far so that it can be safely copied with a caml_stat_alloc _after_ pooling mode has been enabled.
Commit
c3c37e34913a943b071da4bedbc8c503b2afe0b1
by david.allsopp
Fix C library options for win32unix
-lws2_32 and -ladvapi32 are already supplied by default, so they don't need to be in unix.cma/unix.cmxa. However, they do need to be passed when building unix.cmxs, and they were previously acquired via unix.cmxa. Tweak the way LDOPTS is used in Makefile.otherlibs.common (which now is only used for the unix library) so that it's correctly passed to both ocamlopt and ocamlmklib.
Commit
7ef74f3abfe2d18272a71e8127cd82f53290b4e4
by david.allsopp
Fix inclusion of libraries when partial linking
ld -r (certainly in GNU binutils) has an empty search path - co-opt the MSVC search code and always resolve libraries when partial linking, except this time _ignore_ the ones which are missing. This seems to fit the rest of -output-complete-obj, given that the _standard_ C libraries are also omitted (-lm, -lpthread, etc.)
Commit
2d22275db15d0edb871bd4fccffee272aff63e40
by david.allsopp
Stop installing and documenting config_main, etc.
Config_main and Config_boot are built to ensure in the build that utils/config.generated.ml.in and utils/config.fixed.ml are kept in sync (so that the next bootstrap doesn't unexpectedly break). However, because these files were generated in the utils directory, they were picked up both by the install recipe and also when generating API documentation.
It's slightly hairy to remove the wildcards and use filter, because we can easily end up with command lines which are too long (even on Unix), so instead these two modules are now generated in utils/config/
Commit
6df2306446fb9bb94c2d5ff76cfc3e4c5e2a361e
by david.allsopp
Add caml_output_value_with_compat primitive
Alternate version of caml_output_value which returns a boolean indicating if the result was 32-bit compatible and would have succeeded if Compat_32 had been included in the flags.
Commit
7994198c4f172b37250a8b732c29405b061d8738
by david.allsopp
Split Meta into two
All the functions in Meta are now only required by the toplevel, however two of them are still quite tangled up with Symtable. Begin the process of disentanglement by moving those two functions to Symtable and the remaining ones directly to the bytecode toplevel (Dynlink already has its own implementations).
Commit
9a38dc4ebca613e28187744c6d1b05b5870a5523
by david.allsopp
Omit CRCS if caml_reify_bytecode isn't used
Bytecode images (including for -output-obj) now only include the CRCS section if the image actually needs dynamic loading (which is identified by the use of the caml_reify_bytecode primitive which is only used by the toplevel and dynlink).
Commit
508027d42ae2222aa0d543aca9b2a0ccd88ae688
by david.allsopp
Eliminate post-processing of bootstrap artefact
In order to be repeatable, the coreboot cycle routinely has to relink both boot artefacts with a fixed configuration. It's not therefore much more of a stretch to link those artefacts with the required flags to suppress both the header (and RNTM section, if applicable) and debugging information.
Combined with the previous change to suppress CRCS when the image doesn't use dynamic loading, the coreboot cycle can be accomplished without having to post-process the artefacts with stripdebug.
![[Jenkins]](/ocaml/static/db2b9b32/images/svgs/logo.svg){kind=logo}
