Tags: 0x88888888/cloud-hypervisor
Tags
v16.0 This release has been tracked through the [v16.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/19). Improved live migration support ------------------------------- The live migration support inside Cloud Hypervisor has been improved with the addition of the tracking of dirty pages written by the VMM to complement the tracking of dirty pages made by the guest itself. Further the internal state of the VMM now is versioned which allows the safe migration of VMs from one version of the VMM to a newer one. However further testing is required so this should be done with care. See the [live migration documentation](docs/live_migration.md) for more details. Improved `vhost-user` support ----------------------------- When using `vhost-user` to access devices implemented in different processes there is now support for reconnection of those devices in the case of a restart of the backend. In addition it is now possible to operate with the direction of the `vhost-user-net` connection reversed with the server in the VMM and the client in the backend. This is aligns with the default approach recommended by Open vSwitch. ARM64 ACPI and UEFI support --------------------------- Cloud Hypervisor now supports using ACPI and booting from a UEFI image on ARM64. This allows the use of stock OS images without direct kernel boot. Notable bug fixes ----------------- * Activating fewer `virtio-net` queues than advertised is now supported. This appeared when using OVMF with an MQ enabled device (cloud-hypervisor#2578). * When using MQ with `virtio` devices Cloud Hypervisor now enforces a minimum vCPU count which ensures that the user will not see adverse guest performance (cloud-hypervisor#2563). * The KVM clock is now correctly handled during live migration / snapshot & restore. Removed functionality --------------------- The following formerly deprecated features have been removed: * Support for booting with the "LinuxBoot" protocol for ELF and `bzImage` binaries has been deprecated. When using direct boot users should configure their kernel with `CONFIG_PVH=y`. Contributors ------------ Many thanks to everyone who has contributed to our release including some new faces. * Anatol Belski <[email protected]> * Bo Chen <[email protected]> * Dayu Liu <[email protected]> * Henry Wang <[email protected]> * Jiachen Zhang <[email protected]> * Jianyong Wu <[email protected]> * Michael Zhao <[email protected]> * Mikko Ylinen <[email protected]> * Muminul Islam <[email protected]> * Ren Lei <[email protected]> * Rob Bradford <[email protected]> * Sebastien Boeuf <[email protected]> * Wei Liu <[email protected]> * Yi Wang <[email protected]>
v15.0 This release has been tracked through the [v15.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/18). Highlights for `cloud-hypervisor` version v15.0 include: Version numbering and stability guarantees ------------------------------------------ This release is the first in a new version numbering scheme to represent that we believe Cloud Hypervisor is maturing and entering a period of stability. With this new release we are beginning our new stability guarantees: * The API (including command line options) will not be removed or changed in a breaking way without a minimum of 2 releases notice. Where possible warnings will be given about the use of deprecated functionality and the deprecations will be documented in the release notes. * Point releases will be made between individual releases where there are substantial bug fixes or security issues that need to be fixed. Currently the following items are **not** guaranteed across updates: * Snapshot/restore is not supported across different versions * Live migration is not supported across different versions * The following features are considered experimental and may change substantially between releases: TDX, SGX. Network device rate limiting ---------------------------- Building on our existing support for rate limiting block activity the network device also now supports rate limiting. Full details of the controls are in the [IO throttling documentation.](docs/io_throttling.md) Support for runtime control of `virtio-net` guest offload --------------------------------------------------------- The guest is now able to change the offload settings for the `virtio-net` device. As well as providing a useful control this mitigates an issue in the Linux kernel where the guest will attempt to reprogram the offload settings even if they are not advertised as configurable (cloud-hypervisor#2528). `--api-socket` supports file descriptor parameter ------------------------------------------------- The `--api-socket` can now take an `fd=` parameter to specify an existing file descriptor to use. This is particularly beneficial for frameworks that need to programmatically control Cloud Hypervisor. Bug fixes --------- * A workaround has been put in place to mitigate a Linux kernel issues that results in the CPU thread spinning at 100% when using `virtio-pmem` (cloud-hypervisor#2277). * PCI BARs are now correctly aligned removing the need for the guest to reprogram them (cloud-hypervisor#1797,cloud-hypervisor#1798) * Handle TAP interface not being writable within virtio-net (due to the buffer exhaustion on the host) (cloud-hypervisor#2517) * The recommended Linux kernel is now v5.12.0 as it contains a fix that prevents snapshot & restore working (cloud-hypervisor#2535) Deprecations ------------ Deprecated features will be removed in a subsequent release and users should plan to use alternatives * Support for booting with the "LinuxBoot" protocol for ELF and `bzImage` binaries has been deprecated. When using direct boot users should configure their kernel with `CONFIG_PVH=y`. Will be removed in v16.0. Contributors ------------ Many thanks to everyone who has contributed to our release including some new faces. * Alyssa Ross <[email protected]> * Anatol Belski <[email protected]> * Bo Chen <[email protected]> * Gaelan Steele <[email protected]> * Jianyong Wu <[email protected]> * Michael Zhao <[email protected]> * Muminul Islam <[email protected]> * Rob Bradford <[email protected]> * Sebastien Boeuf <[email protected]> * Wei Liu <[email protected]> * William Douglas <[email protected]>
v0.14.1 Bug fix release branched off the v0.14.0 release. The following bugs were fixed in this release: * CPU hotplug on Windows failed due to misreported CPU state information and the lack of HyperV CPUID bit enabled (cloud-hypervisor#2437, cloud-hypervisor#2449, cloud-hypervisor#2436) * A seccomp rule was missing that was triggered on CPU unplug (cloud-hypervisor#2455) * A bounds check in VIRTIO queue validation was erroneously generating DescriptorChainTooShort errors in certain circumstances (cloud-hypervisor#2450, cloud-hypervisor#2424)
v0.14.0 This release has been tracked through the [0.14.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/17). Highlights for `cloud-hypervisor` version 0.14.0 include: Structured event monitoring --------------------------- A new option was added to the VMM `--event-monitor` which reports structured events (JSON) over a file or file descriptor at key events in the lifecycle of the VM. The list of events is limited at the moment but will be further extended over subsequent releases. The events exposed form part of the Cloud Hypervisor API surface. MSHV improvements ----------------- Basic support has been added for running Windows guests atop the MSHV hypervisor as an alternative to KVM and further improvements have been made to the MSHV support. Improved aarch64 platform ------------------------- The aarch64 platform has been enhanced with more devices exposed to the running VM including an enhanced serial UART. Updated hotplug documentation ----------------------------- The documentation for the hotplug support has been updated to reflect the use of the `ch-remote` tool and to include details of `virtio-mem` based hotplug as well as documenting hotplug of paravirtualised and VFIO devices. PTY control for serial and `virtio-console` ------------------------------------------- The `--serial` and `--console` parameters can now direct the console to a PTY allowing programmatic control of the console from another process through the PTY subsystem. Block device rate limiting -------------------------- The block device performance can now be constrained as part of the VM configuration allowing rate limiting. Full details of the controls are in the [IO throttling doumentation.](docs/io_throttling.md) Deprecations ------------ Deprecated features will be removed in a subsequent release and users should plan to use alternatives * Support for booting with the "LinuxBoot" protocol for ELF and `bzImage` binaries has been deprecated. When using direct boot users should configure their kernel with `CONFIG_PVH=y`. Contributors ------------ Many thanks to everyone who has contributed to our 0.14.0 release including some new faces. * Bo Chen <[email protected]> * Henry Wang <[email protected]> * Iggy Jackson <[email protected]> * Jiachen Zhang <[email protected]> * Michael Zhao <[email protected]> * Muminul Islam <[email protected]> * Penny Zheng <[email protected]> * Rob Bradford <[email protected]> * Sebastien Boeuf <[email protected]> * Vineeth Pillai <[email protected]> * Wei Liu <[email protected]> * William Douglas <[email protected]> * Zide Chen <[email protected]>
v0.13.0 This release has been tracked through the [0.13.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/16). Highlights for `cloud-hypervisor` version 0.13.0 include: Wider VFIO device support ------------------------- It is now possible to use Cloud Hypervisor's VFIO support to passthrough PCI devices that do not support MSI or MSI-X and instead rely on INTx interrupts. Most notably this widens the support to most NVIDIA cards with the proprietary drivers. Improve huge page support ------------------------- Through the addition of `hugepage_size` on `--memory` it is now possible to specify the desired size of the huge pages used when allocating the guest memory. The user is required to ensure they have sufficient pages of the desired size in their pool. MACvTAP support --------------- It is now possible to provide file descriptors using the `fd` parameter to `--net` which point at TAP devices that have already been opened by the user. This aids integration with `libvirt` but also permits the use of MACvTAP support. This is documented in dedicated [macvtap documentation](docs/macvtap-bridge.md). VHD disk image support ---------------------- It is now possible to use VHD (fixed) disk images as well as QCOWv2 and raw disk image with Cloud Hypervisor. Improved Virtio device threading -------------------------------- Device threads are now derived from the main VMM thread which allows more restrictive seccomp filters to be applied to them. The threads also have a predictable name derived from the device id. Clean shutdown support via synthetic power button ------------------------------------------------- It is now possible to request that the guest VM shut itself down by triggering a synthetic ACPI power button press from the VMM. If the guest is listening for such an event (e.g. using systemd) then it will process the event and cleanly shut down. This functionality is exposed through the HTTP API and can be triggered via `ch-remote --api-socket=<API socket> power-button`. Contributors ------------ Many thanks to everyone who has contributed to our 0.13.0 release including some new faces. * Bo Chen <[email protected]> * Mikko Ylinen <[email protected]> * Muminul Islam <[email protected]> * Rob Bradford <[email protected]> * Samuel Ortiz <[email protected]> * Sebastien Boeuf <[email protected]> * Vineeth Pillai <[email protected]> * Wei Liu <[email protected]> * William Douglas <[email protected]> * Xie Yongji <[email protected]>
v0.12.0 This release has been tracked through the [0.12.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/15). Highlights for `cloud-hypervisor` version 0.12.0 include: ARM64 enhancements ------------------ The use of `--watchdog` is now fully supported as is the ability to reboot the VM from within the guest when running Cloud Hypervisor on an ARM64 system. Removal of `vhost-user-net` and `vhost-user-block` self spawning ---------------------------------------------------------------- In order to use `vhost-user-net` or `vhost-user-block` backends the user is now responsible for starting the backend and providing the socket for the VMM to use. This functionality was deprecated in the last release and how now been removed. Migration of `vhost-user-fs` backend ------------------------------------ The `vhost-user-fs` backend is no longer included in Cloud Hypervisor and it is instead hosted in [it's own repository](https://gitlab.com/virtio-fs/virtiofsd-rs) Enhanced "info" API ------------------- The `vm.info` HTTP API endpoint has been extended to include the details of the devices used by the VM including any VFIO devices used. Contributors ------------ Many thanks to everyone who has contributed to our 0.12.0 release: * Anatol Belski <[email protected]> * Julio Montes <[email protected]> * Michael Zhao <[email protected]> * Muminul Islam <[email protected]> * Rob Bradford <[email protected]> * Samuel Ortiz <[email protected]> * Wei Liu <[email protected]>
v0.11.0 This release has been tracked through the [0.11.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/14). Highlights for `cloud-hypervisor` version 0.11.0 include: `io_uring` support by default for `virtio-block` ------------------------------------------------ Provided that the host OS supports it (Linux kernel 5.8+) then `io_uring` will be used for a significantly higher performance block device. Windows Guest Support --------------------- This is the first release where we officially support Windows running as a guest. Full details of how to setup the image and run Cloud Hypervisor with a Windows guest can be found in the dedicated [Windows documentation](docs/windows.md). `vhost-user` "Self Spawning" Deprecation ---------------------------------------- Automatically spawning a `vhost-user-net` or `vhost-user-block` backend is now deprecated. Users of this functionality will receive a warning and should make adjustments. The functionality will be removed in the next release. `virtio-mmmio` Removal ---------------------- Support for using the `virtio-mmio` transport, rather than using PCI, has been removed. This has been to simplify the code and significantly reduce the testing burden of the project. Snapshot/Restore support for ARM64 ---------------------------------- When running on the ARM64 architecture snapshot and restore has now been implemented. Improved Linux Boot Time ------------------------ The time to boot the Linux kernel has been significantly improved by the identifying some areas of delays around PCI bus probing, IOAPIC programming and MPTABLE issues. Full details can be seen in cloud-hypervisor#1728. `SIGTERM/SIGINT` Interrupt Signal Handling ------------------------------------------ When the VMM process receives the `SIGTERM` or `SIGINT` signals then it will trigger the VMM process to cleanly deallocate resources before exiting. The guest VM will not be cleanly shutdown but the VMM process will clean up its resources. Default Log Level Changed ------------------------- The default logging level was changed to include warnings which should make it easier to see potential issues. New [logging documentation](docs/logging) was also added. New `--balloon` Parameter Added ------------------------------- Control of the setup of `virtio-balloon` has been moved from `--memory` to its own dedicated parameter. This makes it easier to add more balloon specific controls without overloading `--memory`. Experimental `virtio-watchdog` Support -------------------------------------- Support for using a new `virtio-watchdog` has been added which can be used to have the VMM reboot the guest if the guest userspace fails to ping the watchdog. This is enabled with `--watchdog` and requires kernel support. Notable Bug Fixes ----------------- * MTRR bit was missing from CPUID advertised to guest * "Return" key could not be used under `CMD.EXE` under Windows SAC (cloud-hypervisor#1170) * CPU identification string is now exposed to the guest * `virtio-pmem` with`discard_writes=on` no longer marks the guest memory as read only so avoids excessive VM exits (cloud-hypervisor#1795) * PCI device hotplug after an unplug was fixed (cloud-hypervisor#1802) * When using the ACPI method to resize the guest memory the full reserved size can be used (cloud-hypervisor#1803) * Snapshot and restore followed by a second snapshot and restore now works correctly * Snapshot and restore of VMs with more than 2GiB in one region now work correctly Contributors ------------ Many thanks to everyone who has contributed to our 0.11.0 release including some new faces. * Anatol Belski <[email protected]> * Bo Chen <[email protected]> * Daniel Verkamp <[email protected]> * Henry Wang <[email protected]> * Hui Zhu <[email protected]> * Jiangbo Wu <[email protected]> * Josh Soref <[email protected]> * Julio Montes <[email protected]> * Michael Zhao <[email protected]> * Muminul Islam <[email protected]> * pierwill <[email protected]> * Praveen Paladugu <[email protected]> * Rob Bradford <[email protected]> * Sebastien Boeuf <[email protected]> * Wei Liu <[email protected]>
v0.10.0 This release has been tracked through the [0.10.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/13). Highlights for `cloud-hypervisor` version 0.10.0 include: `virtio-block` Support for Multiple Descriptors ----------------------------------------------- Some `virtio-block` device drivers may generate requests with multiple descriptors and support has been added for those drivers. Memory Zones ------------ Support has been added for fine grained control of memory allocation for the guest. This includes controlling the backing of sections of guest memory, assigning to specific host NUMA nodes and assigning memory and vCPUs to specific memory nodes inside the guest. Full details of this can be found in the [memory documentation](docs/memory.md). `Seccomp` Sandbox Improvements ------------------------------ All the remaining threads and devices are now isolated within their own `seccomp` filters. This provides a layer of sandboxing and enhances the security model of `cloud-hypervisor`. Preliminary KVM HyperV Emulation Control ---------------------------------------- A new option (`kvm_hyperv`) has been added to `--cpus` to provide an option to toggle on KVM's HyperV emulation support. This enables progress towards booting Windows without adding extra emulated devices. Notable Bug Fixes ----------------- - When using `ch-remote` to resize the VM parameter now accepts the standard sizes suffices (cloud-hypervisor#1596) - `cloud-hypervisor` no longer panics when started with `--memory hotplug_method=virtio-mem` and no `hotplug_size` (cloud-hypervisor#1564) - After a reboot memory can remove when using `--memory hotplug_method=virtio-mem` (cloud-hypervisor#1593) - `--version` shows the version for released binaries (cloud-hypervisor#1669) - Errors generated by worker threads for `virtio` devices are now printed out (cloud-hypervisor#1551) Contributors ------------ Many thanks to everyone who has contributed to our 0.10.0 release including some new faces. * Alyssa Ross <[email protected]> * Amey Narkhede <[email protected]> * Anatol Belski <[email protected]> * Bo Chen <[email protected]> * Hui Zhu <[email protected]> * Michael Zhao <[email protected]> * Muminul Islam <[email protected]> * Rob Bradford <[email protected]> * Samuel Ortiz <[email protected]> * Sebastien Boeuf <[email protected]> * Wei Liu <[email protected]>
v0.9.0 This release has been tracked through the 0.9.0 project. Highlights for cloud-hypervisor version 0.9.0 include: io_uring Based Block Device Support ---------------------------------- If the io_uring feature is enabled and the host kernel supports it then io_uring will be used for block devices. This results a very significant performance improvement. Block and Network Device Statistics ----------------------------------- Statistics for activity of the virtio network and block devices is now exposed through a new vm.counters HTTP API entry point. These take the form of simple counters which can be used to observe the activity of the VM. HTTP API Responses ------------------ The HTTP API for adding devices now responds with the name that was assigned to the device as well the PCI BDF. CPU Topology ------------ A topology parameter has been added to --cpus which allows the configuration of the guest CPU topology allowing the user to specify the numbers of sockets, packages per socket, cores per package and threads per core. Release Build Optimization -------------------------- Our release build is now built with LTO (Link Time Optimization) which results in a ~20% reduction in the binary size. Hypervisor Abstraction ---------------------- A new abstraction has been introduced, in the form of a hypervisor crate so as to enable the support of additional hypervisors beyond KVM. Snapshot/Restore Improvements ----------------------------- Multiple improvements have been made to the VM snapshot/restore support that was added in the last release. This includes persisting more vCPU state and in particular preserving the guest paravirtualized clock in order to avoid vCPU hangs inside the guest when running with multiple vCPUs. Virtio Memory Ballooning Support -------------------------------- A virtio-balloon device has been added, controlled through the resize control, which allows the reclamation of host memory by resizing a memory balloon inside the guest. Enhancements to ARM64 Support ----------------------------- The ARM64 support introduced in the last release has been further enhanced with support for using PCI for exposing devices into the guest as well as multiple bug fixes. It also now supports using an initramfs when booting. Intel SGX Support ----------------- The guest can now use Intel SGX if the host supports it. Details can be found in the dedicated SGX documentation. Seccomp Sandbox Improvements ---------------------------- The most frequently used virtio devices are now isolated with their own seccomp filters. It is also now possible to pass --seccomp=log which result in the logging of requests that would have otherwise been denied to further aid development. Notable Bug Fixes ----------------- * Our virtio-vsock implementation has been resynced with the implementation from Firecracker and includes multiple bug fixes. CPU hotplug has been fixed so that it is now possible to add, remove, and re-add vCPUs (cloud-hypervisor#1338) * A workaround is now in place for when KVM reports MSRs available MSRs that are in fact unreadable preventing snapshot/restore from working correctly (cloud-hypervisor#1543). * virtio-mmio based devices are now more widely tested (cloud-hypervisor#275). * Multiple issues have been fixed with virtio device configuration (cloud-hypervisor#1217) * Console input was wrongly consumed by both virtio-console and the serial. (cloud-hypervisor#1521) Contributors ------------ Many thanks to everyone who has contributed to our 0.9.0 release including some new faces. * Anatol Belski [email protected] * Bo Chen [email protected] * Dr. David Alan Gilbert [email protected] * Henry Wang [email protected] * Howard Zhang [email protected] * Hui Zhu [email protected] * Jianyong Wu [email protected] * Jose Carlos Venegas Munoz [email protected] * LiYa'nan [email protected] * Michael Zhao [email protected] * Muminul Islam [email protected] * Praveen Paladugu [email protected] * Ricardo Koller [email protected] * Rob Bradford [email protected] * Samuel Ortiz [email protected] * Sebastien Boeuf [email protected] * Stefano Garzarella [email protected] * Wei Liu [email protected]
v0.8.0 This release has been tracked through the [0.8.0 project](https://github.com/cloud-hypervisor/cloud-hypervisor/projects/10). Highlights for `cloud-hypervisor` version 0.8.0 include: Experimental Snapshot and Restore Support ----------------------------------------- This release includes the first version of the snapshot and restore feature. This allows a VM to be paused and then subsequently snapshotted. At a later point that snapshot may be restored into a new running VM identical to the original VM at the point it was paused. This feature can be used for offline migration from one VM host to another, to allow the upgrading or rebooting of the host machine transparently to the guest or for templating the VM. This is an experimental feature and cannot be used on a VM using passthrough (VFIO) devices. Issues with SMP have also been observed (cloud-hypervisor#1176). Experimental ARM64 Support -------------------------- Included in this release is experimental support for running on ARM64. Currently only `virtio-mmio` devices and a serial port are supported. Full details can be found in the [ARM64 documentation](docs/arm64.md). Support for Using 5-level Paging in Guests ------------------------------------------ If the host supports it the guest is now enabled for 5-level paging (aka LA57). This works when booting the Linux kernel with a vmlinux, bzImage or firmware based boot. However booting an ELF kernel built with `CONFIG_PVH=y` does not work due to current limitations in the PVH boot process. Virtio Device Interrupt Suppression for Network Devices ------------------------------------------------------- With `virtio-net` and `vhost-user-net` devices the guest can suppress interrupts from the VMM by using the `VIRTIO_RING_F_EVENT_IDX` feature. This can lead to an improvement in performance by reducing the number of interrupts the guest must service. `vhost_user_fs` Improvements ---------------------------- The implementation in Cloud Hypervisor of the VirtioFS server now supports sandboxing itself with `seccomp`. Notable Bug Fixes ----------------- * VMs that have not yet been booted can now be deleted (cloud-hypervisor#1110). * By creating the `tap` device ahead of creating the VM it is not required to run the `cloud-hypervisor` binary with `CAP_NET_ADMIN` (cloud-hypervisor#1273). * Block I/O via `virtio-block` or `vhost-user-block` now correctly adheres to the specification and synchronizes to the underlying filesystem as required based on guest feature negotiation. This avoids potential data loss (cloud-hypervisor#399, cloud-hypervisor#1216). * When booting with a large number of vCPUs then the ACPI table would be overwritten by the SMP `MPTABLE`. When compiled with the `acpi` feature the `MPTABLE` will no longer be generated (cloud-hypervisor#1132). * Shutting down VMs that have been paused is now supported (cloud-hypervisor#816). * Created socket files are deleted on shutdown (cloud-hypervisor#1083). * Trying to use passthrough devices (VFIO) will be rejected on `mmio` builds (cloud-hypervisor#751). Command Line and API Changes ---------------------------- This is non exhaustive list of HTTP API and command line changes: * All user visible socket parameters are now consistently called `socket` rather than `sock` in some cases. * The `ch-remote` tool now shows any error message generated by the VMM * The `wce` parameter has been removed from `--disk` as the feature is always offered for negotiation. * `--net` has gained a `host_mac` option that allows the setting of the MAC address for the `tap` device on the host. Contributors ------------ Many thanks to everyone who has contributed to our 0.8.0 release including some new faces. * Anatol Belski <[email protected]> * Arron Wang <[email protected]> * Bo Chen <[email protected]> * Dr. David Alan Gilbert <[email protected]> * Henry Wang <[email protected]> * Hui Zhu <[email protected]> * LiYa'nan <[email protected]> * Michael Zhao <[email protected]> * Rob Bradford <[email protected]> * Samuel Ortiz <[email protected]> * Sebastien Boeuf <[email protected]> * Sergio Lopez <[email protected]>
PreviousNext