Auto merge of #113411 - unikraft:unikraft, r=wesleywiser

Add `x86_64-unikraft-linux-musl` target

This introduces `x86_64-unikraft-linux-musl` as the first Rust target for the [Unikraft] Unikernel Development Kit.

[Unikraft]: https://unikraft.org/

Unikraft imitates Linux and uses musl as libc.
It is extremely configurable, and does not even provide a `poll` implementation or a network stack, unless enabled by the end user who compiles the application.

Our approach for integrating the build process with `rustc` is to hide the build process as well as the actual final linking step behind a linker-shim (`kraftld`, see https://github.com/unikraft/kraftkit/issues/612).

## Tier 3 target policy

> - A tier 3 target must have a designated developer or developers (the "target
>   maintainers") on record to be CCed when issues arise regarding the target.
>   (The mechanism to track and CC such developers may evolve over time.)

I will be the target maintainer.

> - Targets must use naming consistent with any existing targets; for instance, a
>   target for the same CPU or OS as an existing Rust target should use the same
>   name for that CPU or OS. Targets should normally use the same names and
>   naming conventions as used elsewhere in the broader ecosystem beyond Rust
>   (such as in other toolchains), unless they have a very good reason to
>   diverge. Changing the name of a target can be highly disruptive, especially
>   once the target reaches a higher tier, so getting the name right is important
>   even for a tier 3 target.
>   - Target names should not introduce undue confusion or ambiguity unless
>     absolutely necessary to maintain ecosystem compatibility. For example, if
>     the name of the target makes people extremely likely to form incorrect
>     beliefs about what it targets, the name should be changed or augmented to
>     disambiguate it.
>   - If possible, use only letters, numbers, dashes and underscores for the name.
>     Periods (`.`) are known to cause issues in Cargo.

The target name `x86_64-unikraft-linux-musl` was derived from `x86_64-unknown-linux-musl`, setting Unikraft as vendor.
Unikraft exactly imitates Linux + musl.

> - Tier 3 targets may have unusual requirements to build or use, but must not
>   create legal issues or impose onerous legal terms for the Rust project or for
>   Rust developers or users.
>   - The target must not introduce license incompatibilities.
>   - Anything added to the Rust repository must be under the standard Rust
>     license (`MIT OR Apache-2.0`).
>   - The target must not cause the Rust tools or libraries built for any other
>     host (even when supporting cross-compilation to the target) to depend
>     on any new dependency less permissive than the Rust licensing policy. This
>     applies whether the dependency is a Rust crate that would require adding
>     new license exceptions (as specified by the `tidy` tool in the
>     rust-lang/rust repository), or whether the dependency is a native library
>     or binary. In other words, the introduction of the target must not cause a
>     user installing or running a version of Rust or the Rust tools to be
>     subject to any new license requirements.
>   - Compiling, linking, and emitting functional binaries, libraries, or other
>     code for the target (whether hosted on the target itself or cross-compiling
>     from another target) must not depend on proprietary (non-FOSS) libraries.
>     Host tools built for the target itself may depend on the ordinary runtime
>     libraries supplied by the platform and commonly used by other applications
>     built for the target, but those libraries must not be required for code
>     generation for the target; cross-compilation to the target must not require
>     such libraries at all. For instance, `rustc` built for the target may
>     depend on a common proprietary C runtime library or console output library,
>     but must not depend on a proprietary code generation library or code
>     optimization library. Rust's license permits such combinations, but the
>     Rust project has no interest in maintaining such combinations within the
>     scope of Rust itself, even at tier 3.
>   - "onerous" here is an intentionally subjective term. At a minimum, "onerous"
>     legal/licensing terms include but are *not* limited to: non-disclosure
>     requirements, non-compete requirements, contributor license agreements
>     (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms,
>     requirements conditional on the employer or employment of any particular
>     Rust developers, revocable terms, any requirements that create liability
>     for the Rust project or its developers or users, or any requirements that
>     adversely affect the livelihood or prospects of the Rust project or its
>     developers or users.

No dependencies were added to Rust.
Requirements for linking are [Unikraft] and [KraftKit] (both BSD-3-Clause), but none of these are added to Rust.

[KraftKit]: https://github.com/unikraft/kraftkit

> - Neither this policy nor any decisions made regarding targets shall create any
>   binding agreement or estoppel by any party. If any member of an approving
>   Rust team serves as one of the maintainers of a target, or has any legal or
>   employment requirement (explicit or implicit) that might affect their
>   decisions regarding a target, they must recuse themselves from any approval
>   decisions regarding the target's tier status, though they may otherwise
>   participate in discussions.
>   - This requirement does not prevent part or all of this policy from being
>     cited in an explicit contract or work agreement (e.g. to implement or
>     maintain support for a target). This requirement exists to ensure that a
>     developer or team responsible for reviewing and approving a target does not
>     face any legal threats or obligations that would prevent them from freely
>     exercising their judgment in such approval, even if such judgment involves
>     subjective matters or goes beyond the letter of these requirements.

Understood.
I am not a member of a Rust team.

> - Tier 3 targets should attempt to implement as much of the standard libraries
>   as possible and appropriate (`core` for most targets, `alloc` for targets
>   that can support dynamic memory allocation, `std` for targets with an
>   operating system or equivalent layer of system-provided functionality), but
>   may leave some code unimplemented (either unavailable or stubbed out as
>   appropriate), whether because the target makes it impossible to implement or
>   challenging to implement. The authors of pull requests are not obligated to
>   avoid calling any portions of the standard library on the basis of a tier 3
>   target not implementing those portions.

Understood.
`std` is supported.

> - The target must provide documentation for the Rust community explaining how
>   to build for the target, using cross-compilation if possible. If the target
>   supports running binaries, or running tests (even if they do not pass), the
>   documentation must explain how to run such binaries or tests for the target,
>   using emulation if possible or dedicated hardware if necessary.

Building is described in the platform support doc.
It will be updated once proper `kraftld` support has landed.

> - Tier 3 targets must not impose burden on the authors of pull requests, or
>   other developers in the community, to maintain the target. In particular,
>   do not post comments (automated or manual) on a PR that derail or suggest a
>   block on the PR based on a tier 3 target. Do not send automated messages or
>   notifications (via any medium, including via ``@`)` to a PR author or others
>   involved with a PR regarding a tier 3 target, unless they have opted into
>   such messages.
>   - Backlinks such as those generated by the issue/PR tracker when linking to
>     an issue or PR are not considered a violation of this policy, within
>     reason. However, such messages (even on a separate repository) must not
>     generate notifications to anyone involved with a PR who has not requested
>     such notifications.

Understood.

> - Patches adding or updating tier 3 targets must not break any existing tier 2
>   or tier 1 target, and must not knowingly break another tier 3 target without
>   approval of either the compiler team or the maintainers of the other tier 3
>   target.
>   - In particular, this may come up when working on closely related targets,
>     such as variations of the same architecture with different features. Avoid
>     introducing unconditional uses of features that another variation of the
>     target may not have; use conditional compilation or runtime detection, as
>     appropriate, to let each target run code supported by that target.

I don't think this PR breaks anything.

r? compiler-team

compiler/rustc_target/src/spec/mod.rs

@@ -85,6 +85,7 @@
 mod solid_base;
 mod thumb_base;
 mod uefi_msvc_base;
+mod unikraft_linux_musl_base;
 mod vxworks_base;
 mod wasm_base;
 mod windows_gnu_base;
@@ -1421,6 +1422,8 @@ fn $module() {
     ("riscv64gc-unknown-hermit", riscv64gc_unknown_hermit),
     ("x86_64-unknown-hermit", x86_64_unknown_hermit),
 
+    ("x86_64-unikraft-linux-musl", x86_64_unikraft_linux_musl),
+
     ("riscv32i-unknown-none-elf", riscv32i_unknown_none_elf),
     ("riscv32im-unknown-none-elf", riscv32im_unknown_none_elf),
     ("riscv32imc-unknown-none-elf", riscv32imc_unknown_none_elf),

compiler/rustc_target/src/spec/unikraft_linux_musl_base.rs

@@ -0,0 +1,15 @@
+use crate::spec::{cvs, PanicStrategy, RelocModel, TargetOptions};
+
+pub fn opts() -> TargetOptions {
+    TargetOptions {
+        os: "linux".into(),
+        env: "musl".into(),
+        vendor: "unikraft".into(),
+        linker: Some("kraftld".into()),
+        relocation_model: RelocModel::Static,
+        families: cvs!["unix"],
+        has_thread_local: true,
+        panic_strategy: PanicStrategy::Abort,
+        ..Default::default()
+    }
+}

compiler/rustc_target/src/spec/x86_64_unikraft_linux_musl.rs

@@ -0,0 +1,19 @@
+use crate::spec::{Cc, LinkerFlavor, Lld, StackProbeType, Target, TargetOptions};
+
+pub fn target() -> Target {
+    Target {
+        llvm_target: "x86_64-unknown-linux-musl".into(),
+        pointer_width: 64,
+        arch: "x86_64".into(),
+        data_layout: "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+            .into(),
+        options: TargetOptions {
+            cpu: "x86-64".into(),
+            plt_by_default: false,
+            pre_link_args: TargetOptions::link_args(LinkerFlavor::Gnu(Cc::Yes, Lld::No), &["-m64"]),
+            max_atomic_width: Some(64),
+            stack_probes: StackProbeType::X86,
+            ..super::unikraft_linux_musl_base::opts()
+        },
+    }
+}

library/std/src/sys/unix/mod.rs

@@ -110,6 +110,11 @@ unsafe fn sanitize_standard_fds() {
             while libc::poll(pfds.as_mut_ptr(), 3, 0) == -1 {
                 match errno() {
                     libc::EINTR => continue,
+                    #[cfg(target_vendor = "unikraft")]
+                    libc::ENOSYS => {
+                        // Not all configurations of Unikraft enable `LIBPOSIX_EVENT`.
+                        break 'poll;
+                    }
                     libc::EINVAL | libc::EAGAIN | libc::ENOMEM => {
                         // RLIMIT_NOFILE or temporary allocation failures
                         // may be preventing use of poll(), fall back to fcntl
@@ -165,7 +170,14 @@ unsafe fn sanitize_standard_fds() {
     }
 
     unsafe fn reset_sigpipe(#[allow(unused_variables)] sigpipe: u8) {
-        #[cfg(not(any(target_os = "emscripten", target_os = "fuchsia", target_os = "horizon")))]
+        #[cfg(not(any(
+            target_os = "emscripten",
+            target_os = "fuchsia",
+            target_os = "horizon",
+            // Unikraft's `signal` implementation is currently broken:
+            // https://github.com/unikraft/lib-musl/issues/57
+            target_vendor = "unikraft",
+        )))]
         {
             // We don't want to add this as a public type to std, nor do we
             // want to `include!` a file from the compiler (which would break

src/bootstrap/compile.rs

@@ -261,7 +261,7 @@ fn copy_self_contained_objects(
     // to using gcc from a glibc-targeting toolchain for linking.
     // To do that we have to distribute musl startup objects as a part of Rust toolchain
     // and link with them manually in the self-contained mode.
-    if target.contains("musl") {
+    if target.contains("musl") && !target.contains("unikraft") {
         let srcdir = builder.musl_libdir(target).unwrap_or_else(|| {
             panic!("Target {:?} does not have a \"musl-libdir\" key", target.triple)
         });

src/bootstrap/lib.rs

@@ -131,6 +131,8 @@ pub unsafe fn setup(_build: &mut crate::Build) {}
     (Some(Mode::Std), "freebsd13", None),
     (Some(Mode::Std), "backtrace_in_libstd", None),
     /* Extra values not defined in the built-in targets yet, but used in std */
+    // #[cfg(bootstrap)]
+    (Some(Mode::Std), "target_vendor", Some(&["unikraft"])),
     (Some(Mode::Std), "target_env", Some(&["libnx"])),
     // (Some(Mode::Std), "target_os", Some(&[])),
     // #[cfg(bootstrap)] mips32r6, mips64r6

src/bootstrap/sanity.rs

@@ -206,7 +206,7 @@ pub fn check(build: &mut Build) {
         }
 
         // Make sure musl-root is valid
-        if target.contains("musl") {
+        if target.contains("musl") && !target.contains("unikraft") {
             // If this is a native target (host is also musl) and no musl-root is given,
             // fall back to the system toolchain in /usr before giving up
             if build.musl_root(*target).is_none() && build.config.build == *target {

src/doc/rustc/src/SUMMARY.md

@@ -42,6 +42,7 @@
     - [sparc-unknown-none-elf](./platform-support/sparc-unknown-none-elf.md)
     - [*-pc-windows-gnullvm](platform-support/pc-windows-gnullvm.md)
     - [\*-nto-qnx-\*](platform-support/nto-qnx.md)
+    - [*-unikraft-linux-musl](platform-support/unikraft-linux-musl.md)
     - [*-unknown-hermit](platform-support/hermit.md)
     - [\*-unknown-netbsd\*](platform-support/netbsd.md)
     - [*-unknown-openbsd](platform-support/openbsd.md)

src/doc/rustc/src/platform-support.md

@@ -326,6 +326,7 @@ target | std | host | notes
 [`x86_64-pc-windows-gnullvm`](platform-support/pc-windows-gnullvm.md) | ✓ | ✓ |
 `x86_64-pc-windows-msvc` | * |  | 64-bit Windows XP support
 `x86_64-sun-solaris` | ? |  | Deprecated target for 64-bit Solaris 10/11, illumos
+[`x86_64-unikraft-linux-musl`](platform-support/unikraft-linux-musl.md) | ✓ |   | 64-bit Unikraft with musl
 `x86_64-unknown-dragonfly` | ✓ | ✓ | 64-bit DragonFlyBSD
 `x86_64-unknown-haiku` | ✓ | ✓ | 64-bit Haiku
 [`x86_64-unknown-hermit`](platform-support/hermit.md) | ✓ |  | x86_64 Hermit

src/doc/rustc/src/platform-support/unikraft-linux-musl.md

@@ -0,0 +1,67 @@
+# `*-unikraft-linux-musl`
+
+**Tier: 3**
+
+Targets for the [Unikraft] Unikernel Development Kit (with musl).
+
+[Unikraft]: https://unikraft.org/
+
+Target triplets available so far:
+
+- `x86_64-unikraft-linux-musl`
+
+## Target maintainers
+
+- Martin Kröning ([@mkroening](https://github.com/mkroening))
+
+## Requirements
+
+These targets only support cross-compilation.
+The targets do support std.
+
+Unikraft pretends to behave exactly like Linux.
+How much of that functionality is available depends on the individual unikernel configuration.
+For example, the basic Unikraft + musl config does not support `poll` or networking out of the box.
+That functionality requires enabling [`LIBPOSIX_EVENT`] or [lwIP] respectively.
+
+[`LIBPOSIX_EVENT`]: https://github.com/unikraft/unikraft/blob/RELEASE-0.13.1/lib/posix-event/Config.uk
+[lwIP]: https://github.com/unikraft/lib-lwip
+
+The Unikraft targets follow Linux's `extern "C"` calling convention.
+
+For these targets, `rustc` does not perform the final linking step.
+Instead, the Unikraft build system will produce the final Unikernel image for the selected platform (e.g., KVM, Linux user space, and Xen).
+
+## Building the targets
+
+You can build Rust with support for the targets by adding it to the `target` list in `config.toml`:
+
+```toml
+[build]
+build-stage = 1
+target = [ "x86_64-unikraft-linux-musl" ]
+```
+
+## Building Rust programs
+
+Rust does not yet ship pre-compiled artifacts for these targets.
+To compile for these targets, you will either need to build Rust with the targets enabled
+(see “Building the targets” above), or build your own copy of `core` by using `build-std` or similar.
+
+Linking requires a [KraftKit] shim.
+See [unikraft/kraftkit#612] for more information.
+
+[KraftKit]: https://github.com/unikraft/kraftkit
+[unikraft/kraftkit#612]: https://github.com/unikraft/kraftkit/issues/612
+
+## Testing
+
+The targets do support running binaries in the form of unikernel images.
+How the unikernel image is run depends on the specific platform (e.g., KVM, Linux user space, and Xen).
+The targets do not support running the Rust test suite.
+
+## Cross-compilation toolchains and C code
+
+The targets do support C code.
+To build compatible C code, you have to use the same compiler and flags as does the Unikraft build system for your specific configuration.
+The easiest way to achieve that, is to build the C code with the Unikraft build system when building your unikernel image.

src/tools/build-manifest/src/main.rs

@@ -148,6 +148,7 @@
     "x86_64-pc-windows-msvc",
     "x86_64-sun-solaris",
     "x86_64-pc-solaris",
+    "x86_64-unikraft-linux-musl",
     "x86_64-unknown-freebsd",
     "x86_64-unknown-illumos",
     "x86_64-unknown-linux-gnu",