Catcrafts/milos-linux
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

drm for v7.1-rc1

Browse source 
mm:
 - two pass MMU interval notifiers
 - add gpu active/reclaim per-node stat counters
 
 math:
 - provide __KERNEL_DIV_ROUND_CLOSEST() in UAPI
 - implement DIV_ROUND_CLOSEST() with __KERNEL_DIV_ROUND_CLOSEST()
 
 rust:
 - shared tag with driver-core: register macro and io infra
 - core: rework DMA coherent API
 - core: add interop::list to interop with C linked lists
 - core: add more num::Bounded operations
 - core: enable generic_arg_infer and add EMSGSIZE
 - workqueue: add ARef<T> support for work and delayed work
 - add GPU buddy allocator abstraction
 - add DRM shmem GEM helper abstraction
 - allow drm:::Device to dispatch work and delayed work items
   to driver private data
 - add dma_resv_lock helper and raw accessors
 
 core:
 - introduce DRM RAS infrastructure over netlink
 - add connector panel_type property
 - fourcc: add ARM interleaved 64k modifier
 - colorop: add destroy helper
 - suballoc: split into alloc and init helpers
 - mode: provide DRM_ARGB_GET*() macros for reading color components
 
 edid:
 - provide drm_output_color_Format
 
 dma-buf:
 - provide revoke mechanism for shared buffers
 - rename move_notify to invalidate_mappings
 - always enable move_notify
 - protect dma_fence_ops with RCU and improve locking
 - clean pages with helpers
 
 atomic:
 - allocate drm_private_state via callback
 - helper: use system_percpu_wq
 
 buddy:
 - make buddy allocator available to gpu level
 - add kernel-doc for buddy allocator
 - improve aligned allocation
 
 ttm:
 - fix fence signalling
 - improve tests and docs
 - improve handling of gfp_retry_mayfail
 - use per-node stat counters to track memory allocations
 - port pool to use list_lru
 - drop NUMA specific pools
 - make pool shrinker numa aware
 - track allocated pages per numa node
 
 coreboot:
 - cleanup coreboot framebuffer support
 
 sched:
 - fix race condition in drm_sched_fini
 
 pagemap:
 - enable THP support
 - pass pagemap_addr by reference
 
 gem-shmem:
 - Track page accessed/dirty status across mmap/vmap
 
 gpusvm:
 - reenable device to device migration
 - fix unbalanced unclock
 
 bridge:
 - anx7625: Support USB-C plus DT bindings
 - connector: Fix EDID detection
 - dw-hdmi-qp: Support Vendor-Specfic and SDP Infoframes; improve others
 - fsl-ldb: Fix visual artifacts plus related DT property 'enable-termination-resistor'
 - imx8qxp-pixel-link: Improve bridge reference handling
 - lt9611: Support Port-B-only input plus DT bindings
 - tda998x: Support DRM_BRIDGE_ATTACH_NO_CONNECTOR; Clean up
 - Support TH1520 HDMI plus DT bindings
 - waveshare-dsi: Fix register and attach; Support 1..4 DSI lanes plus DT bindings
 - anx7625: Fix USB Type-C handling
 - cdns-mhdp8546-core: Handle HDCP state in bridge atomic_check
 - Support Lontium LT8713SX DP MST bridge plus DT bindings
 - analogix_dp: Use DP helpers for link training
 
 panel:
 - panel-jdi-lt070me05000: Use mipi-dsi multi functions
 - panel-edp: Support Add AUO B116XAT04.1 (HW: 1A); Support CMN N116BCL-EAK (C2); Support FriendlyELEC plus DT changes
 - panel-edp: Fix timings for BOE NV140WUM-N64
 - ilitek-ili9882t: Allow GPIO calls to sleep
 - jadard: Support TAIGUAN XTI05101-01A
 - lxd: Support LXD M9189A plus DT bindings
 - mantix: Fix pixel clock; Clean up
 - motorola: Support Motorola Atrix 4G and Droid X2 plus DT bindings
 - novatek: Support Novatek/Tianma NT37700F plus DT bindings
 - simple: Support EDT ET057023UDBA plus DT bindings; Support Powertip
   PH800480T032-ZHC19 plus DT bindings; Support Waveshare 13.3"
 - novatek-nt36672a: Use mipi_dsi_*_multi() functions
 - panel-edp: Support BOE NV153WUM-N42, CMN N153JCA-ELK, CSW MNF307QS3-2
 - support Himax HX83121A plus DT bindings
 - support JuTouch JT070TM041 plus DT bindings
 - support Samsung S6E8FC0 plus DT bindings
 - himax-hx83102c: support Samsung S6E8FC0 plus DT bindings; support backlight
 - ili9806e: support Rocktech RK050HR345-CT106A plus DT bindings
 - simple: support Tianma TM050RDH03 plus DT bindings
 
 amdgpu:
 - enable DC by default on CIK APUs
 - userq fence ioctl param size fixes
 - set panel_type to OLED for eDP
 - refactor DC i2c code
 - FAMS2 update
 - rework ttm handling to allow multiple engines
 - DC DCE 6.x cleanup
 - DC support for NUTMEG/TRAVIS DP bridge
 - DCN 4.2 support
 - GC12 idle power fix for compute
 - use struct drm_edid in non-DC code
 - enable NV12/P010 support on primary planes
 - support newer IP discovery tables
 - VCN/JPEG 5.0.2 support
 - GC/MES 12.1 updates
 - USERQ fixes
 - add DC idle state manager
 - eDP DSC seamless boot
 
 amdkfd:
 - GC 12.1 updates
 - non 4K page fixes
 
 xe:
 - basic Xe3p_LPG and NVL-P enabling patches
 - allow VM_BIND decompress support
 - add purgeable buffer object support
 - add xe_vm_get_property_ioctl
 - restrict multi-lrc to VCS/VECS engines
 - allow disabling VM overcommit in fault mode
 - dGPU memory optimizations
 - Workaround cleanups and simplification
 - Allow VFs VRAM quote changes using sysfs
 - convert GT stats to per-cpu counters
 - pagefault refactors
 - enable multi-queue on xe3p_xpc
 - disable DCC on PTL
 - make MMIO communication more robust
 - disable D3Cold for BMG on specific platforms
 - vfio: improve FLR sync for Xe VFIO
 
 i915/display:
 - C10/C20/LT PHY PLL divider verification
 - use trans push mechanism to generate PSR frame change on LNL+
 - refactor DP DSC slice config
 - VGA decode refactoring
 - refactor DPT, gen2-4 overlay, masked field register macro helpers
 - refactor stolen memory allocation decisions
 - prepare for UHBR DP tunnels
 - refactor LT PHY PLL to use DPLL framework
 - implement register polling/waiting in display code
 - add shared stepping header between i915 and display
 
 i915:
 - fix potential overflow of shmem scatterlist length
 
 nouveau:
 - provide Z cull info to userspace
 - initial GA100 support
 - shutdown on PCI device shutdown
 
 nova-core:
 - harden GSP command queue
 - add support for large RPCs
 - simplify GSP sequencer and message handling
 - refactor falcon firmware handling
 - convert to new register macro
 - conver to new DMA coherent API
 - use checked arithmetic
 - add debugfs support for gsp-rm log buffers
 - fix aux device registration for multi-GPU
 
 msm:
 - CI:
   - Uprev mesa
   - Restore CI jobs for Qualcomm APQ8016 and APQ8096 devices
 - Core:
   - Switched to of_get_available_child_by_name()
 - DPU:
   - Fixes for DSC panels
   - Fixed brownout because of the frequency / OPP mismatch
   - Quad pipe preparation (not enabled yet)
   - Switched to virtual planes by default
   - Dropped VBIF_NRT support
   - Added support for Eliza platform
   - Reworked alpha handling
   - Switched to correct CWB definitions on Eliza
   - Dropped dummy INTF_0 on MSM8953
   - Corrected INTFs related to DP-MST
 - DP:
   - Removed debug prints looking into PHY internals
 - DSI:
   - Fixes for DSC panels
   - RGB101010 support
   - Support for SC8280XP
   - Moved PHY bindings from display/ to phy/
 - GPU:
   - Preemption support for x2-85 and a840
   - IFPC support for a840
   - SKU detection support for x2-85 and a840
   - Expose AQE support (VK ray-pipeline)
   - Avoid locking in VM_BIND fence signaling path
   - Fix to avoid reclaim in GPU snapshot path
   - Disallow foreign mapping of _NO_SHARE BOs
 - HDMI:
   - Fixed infoframes programming
 - MDP5:
   - Dropped support for MSM8974v1
   - Dropped now unused code for MSM8974 v1 and SDM660 / MSM8998
 
 panthor:
 - add tracepoints for power and IRQs
 - fix fence handling
 - extend timestamp query with flags
 - support various sources for timestamp queries
 
 tyr:
 - fix names and model/versions
 
 rockchip:
 - vop2: use drm logging function
 - rk3576 displayport support
 - support CRTC background color
 
 atmel-hlcdc:
 - support sana5d65 LCD controller
 
 tilcdc:
 - use DT bindings schema
 - use managed DRM interfaces
 - support DRM_BRIDGE_ATTACH_NO_CONNECTOR
 
 verisilicon:
 - support DC8200 + DT bindings
 
 virtgpu:
 - support PRIME import with 3D enabled
 
 komeda:
 - fix integer overflow in AFBC checks
 
 mcde:
 - improve bridge handling
 
 gma500:
 - use drm client buffer for fbdev framebuffer
 
 amdxdna:
 - add sensors ioctls
 - provide NPU power estimate
 - support column utilization sensor
 - allow forcing DMA through IOMMU IOVA
 - support per-BO mem usage queries
 - refactor GEM implementation
 
 ivpu:
 - update boot API to v3.29.4
 - limit per-user number of doorbells/contexts
 - perform engine reset on TDR error
 
 loongson:
 - replace custom code with drm_gem_ttm_dumb_map_offset()
 
 imx:
 - support planes behind the primary plane
 - fix bus-format selection
 
 vkms:
 - support CRTC background color
 
 v3d:
 - improve handling of struct v3d_stats
 
 komeda:
 - support Arm China Linlon D6 plus DT bindings
 
 imagination:
 - improve power-off sequence
 - support context-reset notification from firmware
 
 mediatek:
 - mtk_dsi: enable hs clock during pre-enable
 - Remove all conflicting aperture devices during probe
 - Add support for mt8167 display blocks
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEEKbZHaGwW9KfbeusDHTzWXnEhr4FAmnfMHMACgkQDHTzWXnE
 hr4gEg/+Oaf6KBcvqNKPLwDlNeOvHap1n8oiy7SXvOKN2/KEAu/zGpEciJ7GsSge
 qdqY4xhEfp0JZLrTZiIIzFr38uzkanfOLdF2AQCVrfCRhlO7QLiUDxAAdDZUyINe
 kKLvNunxMwhzwsmRHEDL85cgPkhsxt2ux+tUOYZrEQ/ZbdupNrFw9q5ewmuYzGng
 HY8bsnB0jVwQ9IU/X6h+Xzr/19623/CZyUWJSuY1foKMhHMceyrCmpAFEqjFWn71
 7zNYFlPEQtqa6qtIZXVbJB4mhd7NbmMW6s367xx+Sx+UJDDNfS6ku+hpISwxNuVX
 7fOoEkhQ+ynIcxGkfOi5Q9j2/mV/WL/GEA/IUWfmX8l219WOrKY4w0NtCE4C78r7
 QFGUR6w8Vi97FCP8NuA7Kix4J9eSr/FAzqoG0snAOQbVdaTSBr1hL0PeewD8BRry
 PUkCCh6J7jKA6POt4JZeU6mbJ3AMoOwS9BICi10R1R6EnIKNpKGVpAuYHk4B5+u3
 X5vd1ds+8dJN/etaFYgIbirUocKx6zt9rT5i4/wPZIDPoCgZNofePtPCiJoTcnNN
 PUZUngcWLpftwW+kCUdc4lF1Q7nguQpXVpX0WJiSfqejshUTPXHPlmJV81GoNSHo
 fQMUXIjO5cAX0FKPBakSxxwFnOQFq4aZb6kRBt4lYgt+RJfzo3s=
 =GX7Q
 -----END PGP SIGNATURE-----

Merge tag 'drm-next-2026-04-15' of https://gitlab.freedesktop.org/drm/kernel

Pull drm updates from Dave Airlie:
 "Highlights:
   - new DRM RAS infrastructure using netlink
   - amdgpu: enable DC on CIK APUs, and more IP enablement, and more
     user queue work
   - xe: purgeable BO support, and new hw enablement
   - dma-buf : add revocable operations

  Full summary:

  mm:
   - two-pass MMU interval notifiers
   - add gpu active/reclaim per-node stat counters

  math:
   - provide __KERNEL_DIV_ROUND_CLOSEST() in UAPI
   - implement DIV_ROUND_CLOSEST() with __KERNEL_DIV_ROUND_CLOSEST()

  rust:
   - shared tag with driver-core: register macro and io infra
   - core: rework DMA coherent API
   - core: add interop::list to interop with C linked lists
   - core: add more num::Bounded operations
   - core: enable generic_arg_infer and add EMSGSIZE
   - workqueue: add ARef<T> support for work and delayed work
   - add GPU buddy allocator abstraction
   - add DRM shmem GEM helper abstraction
   - allow drm:::Device to dispatch work and delayed work items
     to driver private data
   - add dma_resv_lock helper and raw accessors

  core:
   - introduce DRM RAS infrastructure over netlink
   - add connector panel_type property
   - fourcc: add ARM interleaved 64k modifier
   - colorop: add destroy helper
   - suballoc: split into alloc and init helpers
   - mode: provide DRM_ARGB_GET*() macros for reading color components

  edid:
   - provide drm_output_color_Format

  dma-buf:
   - provide revoke mechanism for shared buffers
   - rename move_notify to invalidate_mappings
   - always enable move_notify
   - protect dma_fence_ops with RCU and improve locking
   - clean pages with helpers

  atomic:
   - allocate drm_private_state via callback
   - helper: use system_percpu_wq

  buddy:
   - make buddy allocator available to gpu level
   - add kernel-doc for buddy allocator
   - improve aligned allocation

  ttm:
   - fix fence signalling
   - improve tests and docs
   - improve handling of gfp_retry_mayfail
   - use per-node stat counters to track memory allocations
   - port pool to use list_lru
   - drop NUMA specific pools
   - make pool shrinker numa aware
   - track allocated pages per numa node

  coreboot:
   - cleanup coreboot framebuffer support

  sched:
   - fix race condition in drm_sched_fini

  pagemap:
   - enable THP support
   - pass pagemap_addr by reference

  gem-shmem:
   - Track page accessed/dirty status across mmap/vmap

  gpusvm:
   - reenable device to device migration
   - fix unbalanced unclock

  bridge:
   - anx7625: Support USB-C plus DT bindings
   - connector: Fix EDID detection
   - dw-hdmi-qp: Support Vendor-Specfic and SDP Infoframes; improve
     others
   - fsl-ldb: Fix visual artifacts plus related DT property
     'enable-termination-resistor'
   - imx8qxp-pixel-link: Improve bridge reference handling
   - lt9611: Support Port-B-only input plus DT bindings
   - tda998x: Support DRM_BRIDGE_ATTACH_NO_CONNECTOR; Clean up
   - Support TH1520 HDMI plus DT bindings
   - waveshare-dsi: Fix register and attach; Support 1..4 DSI lanes plus
     DT bindings
   - anx7625: Fix USB Type-C handling
   - cdns-mhdp8546-core: Handle HDCP state in bridge atomic_check
   - Support Lontium LT8713SX DP MST bridge plus DT bindings
   - analogix_dp: Use DP helpers for link training

  panel:
   - panel-jdi-lt070me05000: Use mipi-dsi multi functions
   - panel-edp: Support Add AUO B116XAT04.1 (HW: 1A); Support CMN
     N116BCL-EAK (C2); Support FriendlyELEC plus DT changes
   - panel-edp: Fix timings for BOE NV140WUM-N64
   - ilitek-ili9882t: Allow GPIO calls to sleep
   - jadard: Support TAIGUAN XTI05101-01A
   - lxd: Support LXD M9189A plus DT bindings
   - mantix: Fix pixel clock; Clean up
   - motorola: Support Motorola Atrix 4G and Droid X2 plus DT bindings
   - novatek: Support Novatek/Tianma NT37700F plus DT bindings
   - simple: Support EDT ET057023UDBA plus DT bindings; Support Powertip
     PH800480T032-ZHC19 plus DT bindings; Support Waveshare 13.3"
   - novatek-nt36672a: Use mipi_dsi_*_multi() functions
   - panel-edp: Support BOE NV153WUM-N42, CMN N153JCA-ELK, CSW
     MNF307QS3-2
   - support Himax HX83121A plus DT bindings
   - support JuTouch JT070TM041 plus DT bindings
   - support Samsung S6E8FC0 plus DT bindings
   - himax-hx83102c: support Samsung S6E8FC0 plus DT bindings; support
     backlight
   - ili9806e: support Rocktech RK050HR345-CT106A plus DT bindings
   - simple: support Tianma TM050RDH03 plus DT bindings

  amdgpu:
   - enable DC by default on CIK APUs
   - userq fence ioctl param size fixes
   - set panel_type to OLED for eDP
   - refactor DC i2c code
   - FAMS2 update
   - rework ttm handling to allow multiple engines
   - DC DCE 6.x cleanup
   - DC support for NUTMEG/TRAVIS DP bridge
   - DCN 4.2 support
   - GC12 idle power fix for compute
   - use struct drm_edid in non-DC code
   - enable NV12/P010 support on primary planes
   - support newer IP discovery tables
   - VCN/JPEG 5.0.2 support
   - GC/MES 12.1 updates
   - USERQ fixes
   - add DC idle state manager
   - eDP DSC seamless boot

  amdkfd:
   - GC 12.1 updates
   - non 4K page fixes

  xe:
   - basic Xe3p_LPG and NVL-P enabling patches
   - allow VM_BIND decompress support
   - add purgeable buffer object support
   - add xe_vm_get_property_ioctl
   - restrict multi-lrc to VCS/VECS engines
   - allow disabling VM overcommit in fault mode
   - dGPU memory optimizations
   - Workaround cleanups and simplification
   - Allow VFs VRAM quote changes using sysfs
   - convert GT stats to per-cpu counters
   - pagefault refactors
   - enable multi-queue on xe3p_xpc
   - disable DCC on PTL
   - make MMIO communication more robust
   - disable D3Cold for BMG on specific platforms
   - vfio: improve FLR sync for Xe VFIO

  i915/display:
   - C10/C20/LT PHY PLL divider verification
   - use trans push mechanism to generate PSR frame change on LNL+
   - refactor DP DSC slice config
   - VGA decode refactoring
   - refactor DPT, gen2-4 overlay, masked field register macro helpers
   - refactor stolen memory allocation decisions
   - prepare for UHBR DP tunnels
   - refactor LT PHY PLL to use DPLL framework
   - implement register polling/waiting in display code
   - add shared stepping header between i915 and display

  i915:
   - fix potential overflow of shmem scatterlist length

  nouveau:
   - provide Z cull info to userspace
   - initial GA100 support
   - shutdown on PCI device shutdown

  nova-core:
   - harden GSP command queue
   - add support for large RPCs
   - simplify GSP sequencer and message handling
   - refactor falcon firmware handling
   - convert to new register macro
   - conver to new DMA coherent API
   - use checked arithmetic
   - add debugfs support for gsp-rm log buffers
   - fix aux device registration for multi-GPU

  msm:
   - CI:
      - Uprev mesa
      - Restore CI jobs for Qualcomm APQ8016 and APQ8096 devices
   - Core:
      - Switched to of_get_available_child_by_name()
   - DPU:
      - Fixes for DSC panels
      - Fixed brownout because of the frequency / OPP mismatch
      - Quad pipe preparation (not enabled yet)
      - Switched to virtual planes by default
      - Dropped VBIF_NRT support
      - Added support for Eliza platform
      - Reworked alpha handling
      - Switched to correct CWB definitions on Eliza
      - Dropped dummy INTF_0 on MSM8953
      - Corrected INTFs related to DP-MST
   - DP:
      - Removed debug prints looking into PHY internals
   - DSI:
      - Fixes for DSC panels
      - RGB101010 support
      - Support for SC8280XP
      - Moved PHY bindings from display/ to phy/
   - GPU:
      - Preemption support for x2-85 and a840
      - IFPC support for a840
      - SKU detection support for x2-85 and a840
      - Expose AQE support (VK ray-pipeline)
      - Avoid locking in VM_BIND fence signaling path
      - Fix to avoid reclaim in GPU snapshot path
      - Disallow foreign mapping of _NO_SHARE BOs
   - HDMI:
      - Fixed infoframes programming
   - MDP5:
      - Dropped support for MSM8974v1
      - Dropped now unused code for MSM8974 v1 and SDM660 / MSM8998

  panthor:
   - add tracepoints for power and IRQs
   - fix fence handling
   - extend timestamp query with flags
   - support various sources for timestamp queries

  tyr:
   - fix names and model/versions

  rockchip:
   - vop2: use drm logging function
   - rk3576 displayport support
   - support CRTC background color

  atmel-hlcdc:
   - support sana5d65 LCD controller

  tilcdc:
   - use DT bindings schema
   - use managed DRM interfaces
   - support DRM_BRIDGE_ATTACH_NO_CONNECTOR

  verisilicon:
   - support DC8200 + DT bindings

  virtgpu:
   - support PRIME import with 3D enabled

  komeda:
   - fix integer overflow in AFBC checks

  mcde:
   - improve bridge handling

  gma500:
   - use drm client buffer for fbdev framebuffer

  amdxdna:
   - add sensors ioctls
   - provide NPU power estimate
   - support column utilization sensor
   - allow forcing DMA through IOMMU IOVA
   - support per-BO mem usage queries
   - refactor GEM implementation

  ivpu:
   - update boot API to v3.29.4
   - limit per-user number of doorbells/contexts
   - perform engine reset on TDR error

  loongson:
   - replace custom code with drm_gem_ttm_dumb_map_offset()

  imx:
   - support planes behind the primary plane
   - fix bus-format selection

  vkms:
   - support CRTC background color

  v3d:
   - improve handling of struct v3d_stats

  komeda:
   - support Arm China Linlon D6 plus DT bindings

  imagination:
   - improve power-off sequence
   - support context-reset notification from firmware

  mediatek:
   - mtk_dsi: enable hs clock during pre-enable
   - Remove all conflicting aperture devices during probe
   - Add support for mt8167 display blocks"

* tag 'drm-next-2026-04-15' of https://gitlab.freedesktop.org/drm/kernel: (1735 commits)
  drm/ttm/tests: Remove checks from ttm_pool_free_no_dma_alloc
  drm/ttm/tests: fix lru_count ASSERT
  drm/vram: remove DRM_VRAM_MM_FILE_OPERATIONS from docs
  drm/fb-helper: Fix a locking bug in an error path
  dma-fence: correct kernel-doc function parameter @flags
  ttm/pool: track allocated_pages per numa node.
  ttm/pool: make pool shrinker NUMA aware (v2)
  ttm/pool: drop numa specific pools
  ttm/pool: port to list_lru. (v2)
  drm/ttm: use gpu mm stats to track gpu memory allocations. (v4)
  mm: add gpu active/reclaim per-node stat counters (v2)
  gpu: nova-core: fix missing colon in SEC2 boot debug message
  gpu: nova-core: vbios: use from_le_bytes() for PCI ROM header parsing
  gpu: nova-core: bitfield: fix broken Default implementation
  gpu: nova-core: falcon: pad firmware DMA object size to required block alignment
  gpu: nova-core: gsp: fix undefined behavior in command queue code
  drm/shmem_helper: Make sure PMD entries get the writeable upgrade
  accel/ivpu: Trigger recovery on TDR with OS scheduling
  drm/msm: Use of_get_available_child_by_name()
  dt-bindings: display/msm: move DSI PHY bindings to phy/ subdir
  ...
This commit is contained in:
Linus Torvalds 2026-04-15 08:45:00 -07:00
commit 4a57e0913e
1734 changed files with 163040 additions and 28279 deletions
Download patch file Download diff file Expand all files Collapse all files

14
rust/bindings/bindings_helper.h
View file

@ -29,10 +29,12 @@
#include <linux/hrtimer_types.h>
#include <linux/acpi.h>
#include <linux/gpu_buddy.h>
#include <drm/drm_device.h>
#include <drm/drm_drv.h>
#include <drm/drm_file.h>
#include <drm/drm_gem.h>
#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_ioctl.h>
#include <kunit/test.h>
#include <linux/auxiliary_bus.h>
@ -51,6 +53,7 @@
#include <linux/device/faux.h>
#include <linux/dma-direction.h>
#include <linux/dma-mapping.h>
#include <linux/dma-resv.h>
#include <linux/errname.h>
#include <linux/ethtool.h>
#include <linux/fdtable.h>
@ -61,6 +64,7 @@
#include <linux/interrupt.h>
#include <linux/io-pgtable.h>
#include <linux/ioport.h>
#include <linux/iosys-map.h>
#include <linux/jiffies.h>
#include <linux/jump_label.h>
#include <linux/mdio.h>
@ -146,6 +150,16 @@ const vm_flags_t RUST_CONST_HELPER_VM_MIXEDMAP = VM_MIXEDMAP;
const vm_flags_t RUST_CONST_HELPER_VM_HUGEPAGE = VM_HUGEPAGE;
const vm_flags_t RUST_CONST_HELPER_VM_NOHUGEPAGE = VM_NOHUGEPAGE;
#if IS_ENABLED(CONFIG_GPU_BUDDY)
const unsigned long RUST_CONST_HELPER_GPU_BUDDY_RANGE_ALLOCATION = GPU_BUDDY_RANGE_ALLOCATION;
const unsigned long RUST_CONST_HELPER_GPU_BUDDY_TOPDOWN_ALLOCATION = GPU_BUDDY_TOPDOWN_ALLOCATION;
const unsigned long RUST_CONST_HELPER_GPU_BUDDY_CONTIGUOUS_ALLOCATION =
GPU_BUDDY_CONTIGUOUS_ALLOCATION;
const unsigned long RUST_CONST_HELPER_GPU_BUDDY_CLEAR_ALLOCATION = GPU_BUDDY_CLEAR_ALLOCATION;
const unsigned long RUST_CONST_HELPER_GPU_BUDDY_CLEARED = GPU_BUDDY_CLEARED;
const unsigned long RUST_CONST_HELPER_GPU_BUDDY_TRIM_DISABLE = GPU_BUDDY_TRIM_DISABLE;
#endif
#if IS_ENABLED(CONFIG_ANDROID_BINDER_IPC_RUST)
#include "../../drivers/android/binder/rust_binder.h"
#include "../../drivers/android/binder/rust_binder_events.h"

5
rust/helpers/device.c
View file

@ -25,3 +25,8 @@ __rust_helper void rust_helper_dev_set_drvdata(struct device *dev, void *data)
{
dev_set_drvdata(dev, data);
}
__rust_helper const char *rust_helper_dev_name(const struct device *dev)
{
return dev_name(dev);
}

14
rust/helpers/dma-resv.c Normal file
View file

@ -0,0 +1,14 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/dma-resv.h>
__rust_helper
int rust_helper_dma_resv_lock(struct dma_resv *obj, struct ww_acquire_ctx *ctx)
{
return dma_resv_lock(obj, ctx);
}
__rust_helper void rust_helper_dma_resv_unlock(struct dma_resv *obj)
{
dma_resv_unlock(obj);
}

56
rust/helpers/drm.c
View file

@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
#include <drm/drm_gem.h>
#include <drm/drm_gem_shmem_helper.h>
#include <drm/drm_vma_manager.h>
#ifdef CONFIG_DRM
@ -21,4 +22,57 @@ rust_helper_drm_vma_node_offset_addr(struct drm_vma_offset_node *node)
return drm_vma_node_offset_addr(node);
}
#endif
#ifdef CONFIG_DRM_GEM_SHMEM_HELPER
__rust_helper void
rust_helper_drm_gem_shmem_object_free(struct drm_gem_object *obj)
{
return drm_gem_shmem_object_free(obj);
}
__rust_helper void
rust_helper_drm_gem_shmem_object_print_info(struct drm_printer *p, unsigned int indent,
const struct drm_gem_object *obj)
{
drm_gem_shmem_object_print_info(p, indent, obj);
}
__rust_helper int
rust_helper_drm_gem_shmem_object_pin(struct drm_gem_object *obj)
{
return drm_gem_shmem_object_pin(obj);
}
__rust_helper void
rust_helper_drm_gem_shmem_object_unpin(struct drm_gem_object *obj)
{
drm_gem_shmem_object_unpin(obj);
}
__rust_helper struct sg_table *
rust_helper_drm_gem_shmem_object_get_sg_table(struct drm_gem_object *obj)
{
return drm_gem_shmem_object_get_sg_table(obj);
}
__rust_helper int
rust_helper_drm_gem_shmem_object_vmap(struct drm_gem_object *obj,
struct iosys_map *map)
{
return drm_gem_shmem_object_vmap(obj, map);
}
__rust_helper void
rust_helper_drm_gem_shmem_object_vunmap(struct drm_gem_object *obj,
struct iosys_map *map)
{
drm_gem_shmem_object_vunmap(obj, map);
}
__rust_helper int
rust_helper_drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
{
return drm_gem_shmem_object_mmap(obj, vma);
}
#endif /* CONFIG_DRM_GEM_SHMEM_HELPER */
#endif /* CONFIG_DRM */

17
rust/helpers/gpu.c Normal file
View file

@ -0,0 +1,17 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/gpu_buddy.h>
#ifdef CONFIG_GPU_BUDDY
__rust_helper u64 rust_helper_gpu_buddy_block_offset(const struct gpu_buddy_block *block)
{
return gpu_buddy_block_offset(block);
}
__rust_helper unsigned int rust_helper_gpu_buddy_block_order(struct gpu_buddy_block *block)
{
return gpu_buddy_block_order(block);
}
#endif /* CONFIG_GPU_BUDDY */

3
rust/helpers/helpers.c
View file

@ -57,13 +57,16 @@
#include "cred.c"
#include "device.c"
#include "dma.c"
#include "dma-resv.c"
#include "drm.c"
#include "err.c"
#include "irq.c"
#include "fs.c"
#include "gpu.c"
#include "io.c"
#include "jump_label.c"
#include "kunit.c"
#include "list.c"
#include "maple_tree.c"
#include "mm.c"
#include "mutex.c"

17
rust/helpers/list.c Normal file
View file

@ -0,0 +1,17 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Helpers for C circular doubly linked list implementation.
*/
#include <linux/list.h>
__rust_helper void rust_helper_INIT_LIST_HEAD(struct list_head *list)
{
INIT_LIST_HEAD(list);
}
__rust_helper void rust_helper_list_add_tail(struct list_head *new, struct list_head *head)
{
list_add_tail(new, head);
}

15
rust/kernel/device.rs
View file

@ -489,6 +489,17 @@ impl<Ctx: DeviceContext> Device<Ctx> {
// defined as a `#[repr(transparent)]` wrapper around `fwnode_handle`.
Some(unsafe { &*fwnode_handle.cast() })
}
/// Returns the name of the device.
///
/// This is the kobject name of the device, or its initial name if the kobject is not yet
/// available.
#[inline]
pub fn name(&self) -> &CStr {
// SAFETY: By its type invariant `self.as_raw()` is a valid pointer to a `struct device`.
// The returned string is valid for the lifetime of the device.
unsafe { CStr::from_char_ptr(bindings::dev_name(self.as_raw())) }
}
}
// SAFETY: `Device` is a transparent wrapper of a type that doesn't depend on `Device`'s generic
@ -575,7 +586,7 @@ pub struct CoreInternal;
/// The bound context indicates that for the entire duration of the lifetime of a [`Device<Bound>`]
/// reference, the [`Device`] is guaranteed to be bound to a driver.
///
/// Some APIs, such as [`dma::CoherentAllocation`] or [`Devres`] rely on the [`Device`] to be bound,
/// Some APIs, such as [`dma::Coherent`] or [`Devres`] rely on the [`Device`] to be bound,
/// which can be proven with the [`Bound`] device context.
///
/// Any abstraction that can guarantee a scope where the corresponding bus device is bound, should
@ -584,7 +595,7 @@ pub struct CoreInternal;
///
/// [`Devres`]: kernel::devres::Devres
/// [`Devres::access`]: kernel::devres::Devres::access
/// [`dma::CoherentAllocation`]: kernel::dma::CoherentAllocation
/// [`dma::Coherent`]: kernel::dma::Coherent
pub struct Bound;
mod private {

899
rust/kernel/dma.rs
View file

File diff suppressed because it is too large Load diff

87
rust/kernel/drm/device.rs
View file

@ -6,15 +6,34 @@
use crate::{
alloc::allocator::Kmalloc,
bindings, device, drm,
drm::driver::AllocImpl,
bindings, device,
drm::{
self,
driver::AllocImpl, //
},
error::from_err_ptr,
error::Result,
prelude::*,
sync::aref::{ARef, AlwaysRefCounted},
sync::aref::{
ARef,
AlwaysRefCounted, //
},
types::Opaque,
workqueue::{
HasDelayedWork,
HasWork,
Work,
WorkItem, //
},
};
use core::{
alloc::Layout,
mem,
ops::Deref,
ptr::{
self,
NonNull, //
},
};
use core::{alloc::Layout, mem, ops::Deref, ptr, ptr::NonNull};
#[cfg(CONFIG_DRM_LEGACY)]
macro_rules! drm_legacy_fields {
@ -227,3 +246,61 @@ unsafe impl<T: drm::Driver> Send for Device<T> {}
// SAFETY: A `drm::Device` can be shared among threads because all immutable methods are protected
// by the synchronization in `struct drm_device`.
unsafe impl<T: drm::Driver> Sync for Device<T> {}
impl<T, const ID: u64> WorkItem<ID> for Device<T>
where
T: drm::Driver,
T::Data: WorkItem<ID, Pointer = ARef<Device<T>>>,
T::Data: HasWork<Device<T>, ID>,
{
type Pointer = ARef<Device<T>>;
fn run(ptr: ARef<Device<T>>) {
T::Data::run(ptr);
}
}
// SAFETY:
//
// - `raw_get_work` and `work_container_of` return valid pointers by relying on
// `T::Data::raw_get_work` and `container_of`. In particular, `T::Data` is
// stored inline in `drm::Device`, so the `container_of` call is valid.
//
// - The two methods are true inverses of each other: given `ptr: *mut
// Device<T>`, `raw_get_work` will return a `*mut Work<Device<T>, ID>` through
// `T::Data::raw_get_work` and given a `ptr: *mut Work<Device<T>, ID>`,
// `work_container_of` will return a `*mut Device<T>` through `container_of`.
unsafe impl<T, const ID: u64> HasWork<Device<T>, ID> for Device<T>
where
T: drm::Driver,
T::Data: HasWork<Device<T>, ID>,
{
unsafe fn raw_get_work(ptr: *mut Self) -> *mut Work<Device<T>, ID> {
// SAFETY: The caller promises that `ptr` points to a valid `Device<T>`.
let data_ptr = unsafe { &raw mut (*ptr).data };
// SAFETY: `data_ptr` is a valid pointer to `T::Data`.
unsafe { T::Data::raw_get_work(data_ptr) }
}
unsafe fn work_container_of(ptr: *mut Work<Device<T>, ID>) -> *mut Self {
// SAFETY: The caller promises that `ptr` points at a `Work` field in
// `T::Data`.
let data_ptr = unsafe { T::Data::work_container_of(ptr) };
// SAFETY: `T::Data` is stored as the `data` field in `Device<T>`.
unsafe { crate::container_of!(data_ptr, Self, data) }
}
}
// SAFETY: Our `HasWork<T, ID>` implementation returns a `work_struct` that is
// stored in the `work` field of a `delayed_work` with the same access rules as
// the `work_struct` owing to the bound on `T::Data: HasDelayedWork<Device<T>,
// ID>`, which requires that `T::Data::raw_get_work` return a `work_struct` that
// is inside a `delayed_work`.
unsafe impl<T, const ID: u64> HasDelayedWork<Device<T>, ID> for Device<T>
where
T: drm::Driver,
T::Data: HasDelayedWork<Device<T>, ID>,
{
}

10
rust/kernel/drm/driver.rs
View file

@ -5,12 +5,14 @@
//! C header: [`include/drm/drm_drv.h`](srctree/include/drm/drm_drv.h)
use crate::{
bindings, device, devres, drm,
error::{to_result, Result},
bindings,
device,
devres,
drm,
error::to_result,
prelude::*,
sync::aref::ARef,
sync::aref::ARef, //
};
use macros::vtable;
/// Driver use the GEM memory manager. This should be set for all modern drivers.
pub(crate) const FEAT_GEM: u32 = bindings::drm_driver_feature_DRIVER_GEM;

8
rust/kernel/drm/file.rs
View file

@ -4,9 +4,13 @@
//!
//! C header: [`include/drm/drm_file.h`](srctree/include/drm/drm_file.h)
use crate::{bindings, drm, error::Result, prelude::*, types::Opaque};
use crate::{
bindings,
drm,
prelude::*,
types::Opaque, //
};
use core::marker::PhantomData;
use core::pin::Pin;
/// Trait that must be implemented by DRM drivers to represent a DRM File (a client instance).
pub trait DriverFile {

104
rust/kernel/drm/gem/mod.rs
View file

@ -5,15 +5,66 @@
//! C header: [`include/drm/drm_gem.h`](srctree/include/drm/drm_gem.h)
use crate::{
alloc::flags::*,
bindings, drm,
drm::driver::{AllocImpl, AllocOps},
error::{to_result, Result},
bindings,
drm::{
self,
driver::{
AllocImpl,
AllocOps, //
},
},
error::to_result,
prelude::*,
sync::aref::{ARef, AlwaysRefCounted},
sync::aref::{
ARef,
AlwaysRefCounted, //
},
types::Opaque,
};
use core::{ops::Deref, ptr::NonNull};
use core::{
ops::Deref,
ptr::NonNull, //
};
#[cfg(CONFIG_RUST_DRM_GEM_SHMEM_HELPER)]
pub mod shmem;
/// A macro for implementing [`AlwaysRefCounted`] for any GEM object type.
///
/// Since all GEM objects use the same refcounting scheme.
#[macro_export]
macro_rules! impl_aref_for_gem_obj {
(
impl $( <$( $tparam_id:ident ),+> )? for $type:ty
$(
where
$( $bind_param:path : $bind_trait:path ),+
)?
) => {
// SAFETY: All GEM objects are refcounted.
unsafe impl $( <$( $tparam_id ),+> )? $crate::sync::aref::AlwaysRefCounted for $type
where
Self: IntoGEMObject,
$( $( $bind_param : $bind_trait ),+ )?
{
fn inc_ref(&self) {
// SAFETY: The existence of a shared reference guarantees that the refcount is
// non-zero.
unsafe { bindings::drm_gem_object_get(self.as_raw()) };
}
unsafe fn dec_ref(obj: core::ptr::NonNull<Self>) {
// SAFETY: `obj` is a valid pointer to an `Object<T>`.
let obj = unsafe { obj.as_ref() }.as_raw();
// SAFETY: The safety requirements guarantee that the refcount is non-zero.
unsafe { bindings::drm_gem_object_put(obj) };
}
}
};
}
#[cfg_attr(not(CONFIG_RUST_DRM_GEM_SHMEM_HELPER), allow(unused))]
pub(crate) use impl_aref_for_gem_obj;
/// A type alias for retrieving a [`Driver`]s [`DriverFile`] implementation from its
/// [`DriverObject`] implementation.
@ -27,8 +78,15 @@ pub trait DriverObject: Sync + Send + Sized {
/// Parent `Driver` for this object.
type Driver: drm::Driver;
/// The data type to use for passing arguments to [`DriverObject::new`].
type Args;
/// Create a new driver data object for a GEM object of a given size.
fn new(dev: &drm::Device<Self::Driver>, size: usize) -> impl PinInit<Self, Error>;
fn new(
dev: &drm::Device<Self::Driver>,
size: usize,
args: Self::Args,
) -> impl PinInit<Self, Error>;
/// Open a new handle to an existing object, associated with a File.
fn open(_obj: &<Self::Driver as drm::Driver>::Object, _file: &DriverFile<Self>) -> Result {
@ -162,6 +220,18 @@ pub trait BaseObject: IntoGEMObject {
impl<T: IntoGEMObject> BaseObject for T {}
/// Crate-private base operations shared by all GEM object classes.
#[cfg_attr(not(CONFIG_RUST_DRM_GEM_SHMEM_HELPER), expect(unused))]
pub(crate) trait BaseObjectPrivate: IntoGEMObject {
/// Return a pointer to this object's dma_resv.
fn raw_dma_resv(&self) -> *mut bindings::dma_resv {
// SAFETY: `self.as_raw()` always returns a valid pointer to the base DRM GEM object.
unsafe { (*self.as_raw()).resv }
}
}
impl<T: IntoGEMObject> BaseObjectPrivate for T {}
/// A base GEM object.
///
/// # Invariants
@ -195,11 +265,11 @@ impl<T: DriverObject> Object<T> {
};
/// Create a new GEM object.
pub fn new(dev: &drm::Device<T::Driver>, size: usize) -> Result<ARef<Self>> {
pub fn new(dev: &drm::Device<T::Driver>, size: usize, args: T::Args) -> Result<ARef<Self>> {
let obj: Pin<KBox<Self>> = KBox::pin_init(
try_pin_init!(Self {
obj: Opaque::new(bindings::drm_gem_object::default()),
data <- T::new(dev, size),
data <- T::new(dev, size, args),
}),
GFP_KERNEL,
)?;
@ -252,21 +322,7 @@ impl<T: DriverObject> Object<T> {
}
}
// SAFETY: Instances of `Object<T>` are always reference-counted.
unsafe impl<T: DriverObject> crate::sync::aref::AlwaysRefCounted for Object<T> {
fn inc_ref(&self) {
// SAFETY: The existence of a shared reference guarantees that the refcount is non-zero.
unsafe { bindings::drm_gem_object_get(self.as_raw()) };
}
unsafe fn dec_ref(obj: NonNull<Self>) {
// SAFETY: `obj` is a valid pointer to an `Object<T>`.
let obj = unsafe { obj.as_ref() };
// SAFETY: The safety requirements guarantee that the refcount is non-zero.
unsafe { bindings::drm_gem_object_put(obj.as_raw()) }
}
}
impl_aref_for_gem_obj!(impl<T> for Object<T> where T: DriverObject);
impl<T: DriverObject> super::private::Sealed for Object<T> {}

228
rust/kernel/drm/gem/shmem.rs Normal file
View file

@ -0,0 +1,228 @@
// SPDX-License-Identifier: GPL-2.0
//! DRM GEM shmem helper objects
//!
//! C header: [`include/linux/drm/drm_gem_shmem_helper.h`](srctree/include/drm/drm_gem_shmem_helper.h)
// TODO:
// - There are a number of spots here that manually acquire/release the DMA reservation lock using
// dma_resv_(un)lock(). In the future we should add support for ww mutex, expose a method to
// acquire a reference to the WwMutex, and then use that directly instead of the C functions here.
use crate::{
container_of,
drm::{
device,
driver,
gem,
private::Sealed, //
},
error::to_result,
prelude::*,
types::{
ARef,
Opaque, //
}, //
};
use core::{
ops::{
Deref,
DerefMut, //
},
ptr::NonNull,
};
use gem::{
BaseObjectPrivate,
DriverObject,
IntoGEMObject, //
};
/// A struct for controlling the creation of shmem-backed GEM objects.
///
/// This is used with [`Object::new()`] to control various properties that can only be set when
/// initially creating a shmem-backed GEM object.
#[derive(Default)]
pub struct ObjectConfig<'a, T: DriverObject> {
/// Whether to set the write-combine map flag.
pub map_wc: bool,
/// Reuse the DMA reservation from another GEM object.
///
/// The newly created [`Object`] will hold an owned refcount to `parent_resv_obj` if specified.
pub parent_resv_obj: Option<&'a Object<T>>,
}
/// A shmem-backed GEM object.
///
/// # Invariants
///
/// `obj` contains a valid initialized `struct drm_gem_shmem_object` for the lifetime of this
/// object.
#[repr(C)]
#[pin_data]
pub struct Object<T: DriverObject> {
#[pin]
obj: Opaque<bindings::drm_gem_shmem_object>,
/// Parent object that owns this object's DMA reservation object.
parent_resv_obj: Option<ARef<Object<T>>>,
#[pin]
inner: T,
}
super::impl_aref_for_gem_obj!(impl<T> for Object<T> where T: DriverObject);
// SAFETY: All GEM objects are thread-safe.
unsafe impl<T: DriverObject> Send for Object<T> {}
// SAFETY: All GEM objects are thread-safe.
unsafe impl<T: DriverObject> Sync for Object<T> {}
impl<T: DriverObject> Object<T> {
/// `drm_gem_object_funcs` vtable suitable for GEM shmem objects.
const VTABLE: bindings::drm_gem_object_funcs = bindings::drm_gem_object_funcs {
free: Some(Self::free_callback),
open: Some(super::open_callback::<T>),
close: Some(super::close_callback::<T>),
print_info: Some(bindings::drm_gem_shmem_object_print_info),
export: None,
pin: Some(bindings::drm_gem_shmem_object_pin),
unpin: Some(bindings::drm_gem_shmem_object_unpin),
get_sg_table: Some(bindings::drm_gem_shmem_object_get_sg_table),
vmap: Some(bindings::drm_gem_shmem_object_vmap),
vunmap: Some(bindings::drm_gem_shmem_object_vunmap),
mmap: Some(bindings::drm_gem_shmem_object_mmap),
status: None,
rss: None,
#[allow(unused_unsafe, reason = "Safe since Rust 1.82.0")]
// SAFETY: `drm_gem_shmem_vm_ops` is a valid, static const on the C side.
vm_ops: unsafe { &raw const bindings::drm_gem_shmem_vm_ops },
evict: None,
};
/// Return a raw pointer to the embedded drm_gem_shmem_object.
fn as_raw_shmem(&self) -> *mut bindings::drm_gem_shmem_object {
self.obj.get()
}
/// Create a new shmem-backed DRM object of the given size.
///
/// Additional config options can be specified using `config`.
pub fn new(
dev: &device::Device<T::Driver>,
size: usize,
config: ObjectConfig<'_, T>,
args: T::Args,
) -> Result<ARef<Self>> {
let new: Pin<KBox<Self>> = KBox::try_pin_init(
try_pin_init!(Self {
obj <- Opaque::init_zeroed(),
parent_resv_obj: config.parent_resv_obj.map(|p| p.into()),
inner <- T::new(dev, size, args),
}),
GFP_KERNEL,
)?;
// SAFETY: `obj.as_raw()` is guaranteed to be valid by the initialization above.
unsafe { (*new.as_raw()).funcs = &Self::VTABLE };
// SAFETY: The arguments are all valid via the type invariants.
to_result(unsafe { bindings::drm_gem_shmem_init(dev.as_raw(), new.as_raw_shmem(), size) })?;
// SAFETY: We never move out of `self`.
let new = KBox::into_raw(unsafe { Pin::into_inner_unchecked(new) });
// SAFETY: We're taking over the owned refcount from `drm_gem_shmem_init`.
let obj = unsafe { ARef::from_raw(NonNull::new_unchecked(new)) };
// Start filling out values from `config`
if let Some(parent_resv) = config.parent_resv_obj {
// SAFETY: We have yet to expose the new gem object outside of this function, so it is
// safe to modify this field.
unsafe { (*obj.obj.get()).base.resv = parent_resv.raw_dma_resv() };
}
// SAFETY: We have yet to expose this object outside of this function, so we're guaranteed
// to have exclusive access - thus making this safe to hold a mutable reference to.
let shmem = unsafe { &mut *obj.as_raw_shmem() };
shmem.set_map_wc(config.map_wc);
Ok(obj)
}
/// Returns the `Device` that owns this GEM object.
pub fn dev(&self) -> &device::Device<T::Driver> {
// SAFETY: `dev` will have been initialized in `Self::new()` by `drm_gem_shmem_init()`.
unsafe { device::Device::from_raw((*self.as_raw()).dev) }
}
extern "C" fn free_callback(obj: *mut bindings::drm_gem_object) {
// SAFETY:
// - DRM always passes a valid gem object here
// - We used drm_gem_shmem_create() in our create_gem_object callback, so we know that
// `obj` is contained within a drm_gem_shmem_object
let this = unsafe { container_of!(obj, bindings::drm_gem_shmem_object, base) };
// SAFETY:
// - We're in free_callback - so this function is safe to call.
// - We won't be using the gem resources on `this` after this call.
unsafe { bindings::drm_gem_shmem_release(this) };
// SAFETY:
// - We verified above that `obj` is valid, which makes `this` valid
// - This function is set in AllocOps, so we know that `this` is contained within a
// `Object<T>`
let this = unsafe { container_of!(Opaque::cast_from(this), Self, obj) }.cast_mut();
// SAFETY: We're recovering the Kbox<> we created in gem_create_object()
let _ = unsafe { KBox::from_raw(this) };
}
}
impl<T: DriverObject> Deref for Object<T> {
type Target = T;
fn deref(&self) -> &Self::Target {
&self.inner
}
}
impl<T: DriverObject> DerefMut for Object<T> {
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.inner
}
}
impl<T: DriverObject> Sealed for Object<T> {}
impl<T: DriverObject> gem::IntoGEMObject for Object<T> {
fn as_raw(&self) -> *mut bindings::drm_gem_object {
// SAFETY:
// - Our immutable reference is proof that this is safe to dereference.
// - `obj` is always a valid drm_gem_shmem_object via our type invariants.
unsafe { &raw mut (*self.obj.get()).base }
}
unsafe fn from_raw<'a>(obj: *mut bindings::drm_gem_object) -> &'a Object<T> {
// SAFETY: The safety contract of from_gem_obj() guarantees that `obj` is contained within
// `Self`
unsafe {
let obj = Opaque::cast_from(container_of!(obj, bindings::drm_gem_shmem_object, base));
&*container_of!(obj, Object<T>, obj)
}
}
}
impl<T: DriverObject> driver::AllocImpl for Object<T> {
type Driver = T::Driver;
const ALLOC_OPS: driver::AllocOps = driver::AllocOps {
gem_create_object: None,
prime_handle_to_fd: None,
prime_fd_to_handle: None,
gem_prime_import: None,
gem_prime_import_sg_table: Some(bindings::drm_gem_shmem_prime_import_sg_table),
dumb_create: Some(bindings::drm_gem_shmem_dumb_create),
dumb_map_offset: None,
};
}

1
rust/kernel/error.rs
View file

@ -67,6 +67,7 @@ pub mod code {
declare_err!(EDOM, "Math argument out of domain of func.");
declare_err!(ERANGE, "Math result not representable.");
declare_err!(EOVERFLOW, "Value too large for defined data type.");
declare_err!(EMSGSIZE, "Message too long.");
declare_err!(ETIMEDOUT, "Connection timed out.");
declare_err!(ERESTARTSYS, "Restart the system call.");
declare_err!(ERESTARTNOINTR, "System call was interrupted by a signal and will be restarted.");

6
rust/kernel/gpu.rs Normal file
View file

@ -0,0 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
//! GPU subsystem abstractions.
#[cfg(CONFIG_GPU_BUDDY = "y")]
pub mod buddy;

614
rust/kernel/gpu/buddy.rs Normal file
View file

@ -0,0 +1,614 @@
// SPDX-License-Identifier: GPL-2.0
//! GPU buddy allocator bindings.
//!
//! C header: [`include/linux/gpu_buddy.h`](srctree/include/linux/gpu_buddy.h)
//!
//! This module provides Rust abstractions over the Linux kernel's GPU buddy
//! allocator, which implements a binary buddy memory allocator.
//!
//! The buddy allocator manages a contiguous address space and allocates blocks
//! in power-of-two sizes, useful for GPU physical memory management.
//!
//! # Examples
//!
//! Create a buddy allocator and perform a basic range allocation:
//!
//! ```
//! use kernel::{
//! gpu::buddy::{
//! GpuBuddy,
//! GpuBuddyAllocFlags,
//! GpuBuddyAllocMode,
//! GpuBuddyParams, //
//! },
//! prelude::*,
//! ptr::Alignment,
//! sizes::*, //
//! };
//!
//! // Create a 1GB buddy allocator with 4KB minimum chunk size.
//! let buddy = GpuBuddy::new(GpuBuddyParams {
//! base_offset: 0,
//! size: SZ_1G as u64,
//! chunk_size: Alignment::new::<SZ_4K>(),
//! })?;
//!
//! assert_eq!(buddy.size(), SZ_1G as u64);
//! assert_eq!(buddy.chunk_size(), Alignment::new::<SZ_4K>());
//! let initial_free = buddy.avail();
//!
//! // Allocate 16MB. Block lands at the top of the address range.
//! let allocated = KBox::pin_init(
//! buddy.alloc_blocks(
//! GpuBuddyAllocMode::Simple,
//! SZ_16M as u64,
//! Alignment::new::<SZ_16M>(),
//! GpuBuddyAllocFlags::default(),
//! ),
//! GFP_KERNEL,
//! )?;
//! assert_eq!(buddy.avail(), initial_free - SZ_16M as u64);
//!
//! let block = allocated.iter().next().expect("expected one block");
//! assert_eq!(block.offset(), (SZ_1G - SZ_16M) as u64);
//! assert_eq!(block.order(), 12); // 2^12 pages = 16MB
//! assert_eq!(block.size(), SZ_16M as u64);
//! assert_eq!(allocated.iter().count(), 1);
//!
//! // Dropping the allocation returns the range to the buddy allocator.
//! drop(allocated);
//! assert_eq!(buddy.avail(), initial_free);
//! # Ok::<(), Error>(())
//! ```
//!
//! Top-down allocation allocates from the highest addresses:
//!
//! ```
//! # use kernel::{
//! # gpu::buddy::{GpuBuddy, GpuBuddyAllocMode, GpuBuddyAllocFlags, GpuBuddyParams},
//! # prelude::*,
//! # ptr::Alignment,
//! # sizes::*, //
//! # };
//! # let buddy = GpuBuddy::new(GpuBuddyParams {
//! # base_offset: 0,
//! # size: SZ_1G as u64,
//! # chunk_size: Alignment::new::<SZ_4K>(),
//! # })?;
//! # let initial_free = buddy.avail();
//! let topdown = KBox::pin_init(
//! buddy.alloc_blocks(
//! GpuBuddyAllocMode::TopDown,
//! SZ_16M as u64,
//! Alignment::new::<SZ_16M>(),
//! GpuBuddyAllocFlags::default(),
//! ),
//! GFP_KERNEL,
//! )?;
//! assert_eq!(buddy.avail(), initial_free - SZ_16M as u64);
//!
//! let block = topdown.iter().next().expect("expected one block");
//! assert_eq!(block.offset(), (SZ_1G - SZ_16M) as u64);
//! assert_eq!(block.order(), 12);
//! assert_eq!(block.size(), SZ_16M as u64);
//!
//! // Dropping the allocation returns the range to the buddy allocator.
//! drop(topdown);
//! assert_eq!(buddy.avail(), initial_free);
//! # Ok::<(), Error>(())
//! ```
//!
//! Non-contiguous allocation can fill fragmented memory by returning multiple
//! blocks:
//!
//! ```
//! # use kernel::{
//! # gpu::buddy::{
//! # GpuBuddy, GpuBuddyAllocFlags, GpuBuddyAllocMode, GpuBuddyParams,
//! # },
//! # prelude::*,
//! # ptr::Alignment,
//! # sizes::*, //
//! # };
//! # let buddy = GpuBuddy::new(GpuBuddyParams {
//! # base_offset: 0,
//! # size: SZ_1G as u64,
//! # chunk_size: Alignment::new::<SZ_4K>(),
//! # })?;
//! # let initial_free = buddy.avail();
//! // Create fragmentation by allocating 4MB blocks at [0,4M) and [8M,12M).
//! let frag1 = KBox::pin_init(
//! buddy.alloc_blocks(
//! GpuBuddyAllocMode::Range(0..SZ_4M as u64),
//! SZ_4M as u64,
//! Alignment::new::<SZ_4M>(),
//! GpuBuddyAllocFlags::default(),
//! ),
//! GFP_KERNEL,
//! )?;
//! assert_eq!(buddy.avail(), initial_free - SZ_4M as u64);
//!
//! let frag2 = KBox::pin_init(
//! buddy.alloc_blocks(
//! GpuBuddyAllocMode::Range(SZ_8M as u64..(SZ_8M + SZ_4M) as u64),
//! SZ_4M as u64,
//! Alignment::new::<SZ_4M>(),
//! GpuBuddyAllocFlags::default(),
//! ),
//! GFP_KERNEL,
//! )?;
//! assert_eq!(buddy.avail(), initial_free - SZ_8M as u64);
//!
//! // Allocate 8MB, this returns 2 blocks from the holes.
//! let fragmented = KBox::pin_init(
//! buddy.alloc_blocks(
//! GpuBuddyAllocMode::Range(0..SZ_16M as u64),
//! SZ_8M as u64,
//! Alignment::new::<SZ_4M>(),
//! GpuBuddyAllocFlags::default(),
//! ),
//! GFP_KERNEL,
//! )?;
//! assert_eq!(buddy.avail(), initial_free - SZ_16M as u64);
//!
//! let (mut count, mut total) = (0u32, 0u64);
//! for block in fragmented.iter() {
//! assert_eq!(block.size(), SZ_4M as u64);
//! total += block.size();
//! count += 1;
//! }
//! assert_eq!(total, SZ_8M as u64);
//! assert_eq!(count, 2);
//! # Ok::<(), Error>(())
//! ```
//!
//! Contiguous allocation fails when only fragmented space is available:
//!
//! ```
//! # use kernel::{
//! # gpu::buddy::{
//! # GpuBuddy, GpuBuddyAllocFlag, GpuBuddyAllocFlags, GpuBuddyAllocMode, GpuBuddyParams,
//! # },
//! # prelude::*,
//! # ptr::Alignment,
//! # sizes::*, //
//! # };
//! // Create a small 16MB buddy allocator with fragmented memory.
//! let small = GpuBuddy::new(GpuBuddyParams {
//! base_offset: 0,
//! size: SZ_16M as u64,
//! chunk_size: Alignment::new::<SZ_4K>(),
//! })?;
//!
//! let _hole1 = KBox::pin_init(
//! small.alloc_blocks(
//! GpuBuddyAllocMode::Range(0..SZ_4M as u64),
//! SZ_4M as u64,
//! Alignment::new::<SZ_4M>(),
//! GpuBuddyAllocFlags::default(),
//! ),
//! GFP_KERNEL,
//! )?;
//!
//! let _hole2 = KBox::pin_init(
//! small.alloc_blocks(
//! GpuBuddyAllocMode::Range(SZ_8M as u64..(SZ_8M + SZ_4M) as u64),
//! SZ_4M as u64,
//! Alignment::new::<SZ_4M>(),
//! GpuBuddyAllocFlags::default(),
//! ),
//! GFP_KERNEL,
//! )?;
//!
//! // 8MB contiguous should fail, only two non-contiguous 4MB holes exist.
//! let result = KBox::pin_init(
//! small.alloc_blocks(
//! GpuBuddyAllocMode::Simple,
//! SZ_8M as u64,
//! Alignment::new::<SZ_4M>(),
//! GpuBuddyAllocFlag::Contiguous,
//! ),
//! GFP_KERNEL,
//! );
//! assert!(result.is_err());
//! # Ok::<(), Error>(())
//! ```
use core::ops::Range;
use crate::{
bindings,
clist_create,
error::to_result,
interop::list::CListHead,
new_mutex,
prelude::*,
ptr::Alignment,
sync::{
lock::mutex::MutexGuard,
Arc,
Mutex, //
},
types::Opaque, //
};
/// Allocation mode for the GPU buddy allocator.
///
/// The mode determines the primary allocation strategy. Modes are mutually
/// exclusive: an allocation is either simple, range-constrained, or top-down.
///
/// Orthogonal modifier flags (e.g., contiguous, clear) are specified separately
/// via [`GpuBuddyAllocFlags`].
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum GpuBuddyAllocMode {
/// Simple allocation without constraints.
Simple,
/// Range-based allocation within the given address range.
Range(Range<u64>),
/// Allocate from top of address space downward.
TopDown,
}
impl GpuBuddyAllocMode {
/// Returns the C flags corresponding to the allocation mode.
fn as_flags(&self) -> usize {
match self {
Self::Simple => 0,
Self::Range(_) => bindings::GPU_BUDDY_RANGE_ALLOCATION,
Self::TopDown => bindings::GPU_BUDDY_TOPDOWN_ALLOCATION,
}
}
/// Extracts the range start/end, defaulting to `(0, 0)` for non-range modes.
fn range(&self) -> (u64, u64) {
match self {
Self::Range(range) => (range.start, range.end),
_ => (0, 0),
}
}
}
crate::impl_flags!(
/// Modifier flags for GPU buddy allocation.
///
/// These flags can be combined with any [`GpuBuddyAllocMode`] to control
/// additional allocation behavior.
#[derive(Clone, Copy, Default, PartialEq, Eq)]
pub struct GpuBuddyAllocFlags(usize);
/// Individual modifier flag for GPU buddy allocation.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum GpuBuddyAllocFlag {
/// Allocate physically contiguous blocks.
Contiguous = bindings::GPU_BUDDY_CONTIGUOUS_ALLOCATION,
/// Request allocation from cleared (zeroed) memory.
Clear = bindings::GPU_BUDDY_CLEAR_ALLOCATION,
/// Disable trimming of partially used blocks.
TrimDisable = bindings::GPU_BUDDY_TRIM_DISABLE,
}
);
/// Parameters for creating a GPU buddy allocator.
pub struct GpuBuddyParams {
/// Base offset (in bytes) where the managed memory region starts.
/// Allocations will be offset by this value.
pub base_offset: u64,
/// Total size (in bytes) of the address space managed by the allocator.
pub size: u64,
/// Minimum allocation unit / chunk size; must be >= 4KB.
pub chunk_size: Alignment,
}
/// Inner structure holding the actual buddy allocator.
///
/// # Synchronization
///
/// The C `gpu_buddy` API requires synchronization (see `include/linux/gpu_buddy.h`).
/// Internal locking ensures all allocator and free operations are properly
/// synchronized, preventing races between concurrent allocations and the
/// freeing that occurs when [`AllocatedBlocks`] is dropped.
///
/// # Invariants
///
/// The inner [`Opaque`] contains an initialized buddy allocator.
#[pin_data(PinnedDrop)]
struct GpuBuddyInner {
#[pin]
inner: Opaque<bindings::gpu_buddy>,
// TODO: Replace `Mutex<()>` with `Mutex<Opaque<..>>` once `Mutex::new()`
// accepts `impl PinInit<T>`.
#[pin]
lock: Mutex<()>,
/// Cached creation parameters (do not change after init).
params: GpuBuddyParams,
}
impl GpuBuddyInner {
/// Create a pin-initializer for the buddy allocator.
fn new(params: GpuBuddyParams) -> impl PinInit<Self, Error> {
let size = params.size;
let chunk_size = params.chunk_size;
// INVARIANT: `gpu_buddy_init` returns 0 on success, at which point the
// `gpu_buddy` structure is initialized and ready for use with all
// `gpu_buddy_*` APIs. `try_pin_init!` only completes if all fields succeed,
// so the invariant holds when construction finishes.
try_pin_init!(Self {
inner <- Opaque::try_ffi_init(|ptr| {
// SAFETY: `ptr` points to valid uninitialized memory from the pin-init
// infrastructure. `gpu_buddy_init` will initialize the structure.
to_result(unsafe {
bindings::gpu_buddy_init(ptr, size, chunk_size.as_usize() as u64)
})
}),
lock <- new_mutex!(()),
params,
})
}
/// Lock the mutex and return a guard for accessing the allocator.
fn lock(&self) -> GpuBuddyGuard<'_> {
GpuBuddyGuard {
inner: self,
_guard: self.lock.lock(),
}
}
}
#[pinned_drop]
impl PinnedDrop for GpuBuddyInner {
fn drop(self: Pin<&mut Self>) {
let guard = self.lock();
// SAFETY: Per the type invariant, `inner` contains an initialized
// allocator. `guard` provides exclusive access.
unsafe { bindings::gpu_buddy_fini(guard.as_raw()) };
}
}
// SAFETY: `GpuBuddyInner` can be sent between threads.
unsafe impl Send for GpuBuddyInner {}
// SAFETY: `GpuBuddyInner` is `Sync` because `GpuBuddyInner::lock`
// serializes all access to the C allocator, preventing data races.
unsafe impl Sync for GpuBuddyInner {}
/// Guard that proves the lock is held, enabling access to the allocator.
///
/// The `_guard` holds the lock for the duration of this guard's lifetime.
struct GpuBuddyGuard<'a> {
inner: &'a GpuBuddyInner,
_guard: MutexGuard<'a, ()>,
}
impl GpuBuddyGuard<'_> {
/// Get a raw pointer to the underlying C `gpu_buddy` structure.
fn as_raw(&self) -> *mut bindings::gpu_buddy {
self.inner.inner.get()
}
}
/// GPU buddy allocator instance.
///
/// This structure wraps the C `gpu_buddy` allocator using reference counting.
/// The allocator is automatically cleaned up when all references are dropped.
///
/// Refer to the module-level documentation for usage examples.
pub struct GpuBuddy(Arc<GpuBuddyInner>);
impl GpuBuddy {
/// Create a new buddy allocator.
///
/// The allocator manages a contiguous address space of the given size, with the
/// specified minimum allocation unit (chunk_size must be at least 4KB).
pub fn new(params: GpuBuddyParams) -> Result<Self> {
Arc::pin_init(GpuBuddyInner::new(params), GFP_KERNEL).map(Self)
}
/// Get the base offset for allocations.
pub fn base_offset(&self) -> u64 {
self.0.params.base_offset
}
/// Get the chunk size (minimum allocation unit).
pub fn chunk_size(&self) -> Alignment {
self.0.params.chunk_size
}
/// Get the total managed size.
pub fn size(&self) -> u64 {
self.0.params.size
}
/// Get the available (free) memory in bytes.
pub fn avail(&self) -> u64 {
let guard = self.0.lock();
// SAFETY: Per the type invariant, `inner` contains an initialized allocator.
// `guard` provides exclusive access.
unsafe { (*guard.as_raw()).avail }
}
/// Allocate blocks from the buddy allocator.
///
/// Returns a pin-initializer for [`AllocatedBlocks`].
pub fn alloc_blocks(
&self,
mode: GpuBuddyAllocMode,
size: u64,
min_block_size: Alignment,
flags: impl Into<GpuBuddyAllocFlags>,
) -> impl PinInit<AllocatedBlocks, Error> {
let buddy_arc = Arc::clone(&self.0);
let (start, end) = mode.range();
let mode_flags = mode.as_flags();
let modifier_flags = flags.into();
// Create pin-initializer that initializes list and allocates blocks.
try_pin_init!(AllocatedBlocks {
buddy: buddy_arc,
list <- CListHead::new(),
_: {
// Reject zero-sized or inverted ranges.
if let GpuBuddyAllocMode::Range(range) = &mode {
if range.is_empty() {
Err::<(), Error>(EINVAL)?;
}
}
// Lock while allocating to serialize with concurrent frees.
let guard = buddy.lock();
// SAFETY: Per the type invariant, `inner` contains an initialized
// allocator. `guard` provides exclusive access.
to_result(unsafe {
bindings::gpu_buddy_alloc_blocks(
guard.as_raw(),
start,
end,
size,
min_block_size.as_usize() as u64,
list.as_raw(),
mode_flags | usize::from(modifier_flags),
)
})?
}
})
}
}
/// Allocated blocks from the buddy allocator with automatic cleanup.
///
/// This structure owns a list of allocated blocks and ensures they are
/// automatically freed when dropped. Use `iter()` to iterate over all
/// allocated blocks.
///
/// # Invariants
///
/// - `list` is an initialized, valid list head containing allocated blocks.
#[pin_data(PinnedDrop)]
pub struct AllocatedBlocks {
#[pin]
list: CListHead,
buddy: Arc<GpuBuddyInner>,
}
impl AllocatedBlocks {
/// Check if the block list is empty.
pub fn is_empty(&self) -> bool {
// An empty list head points to itself.
!self.list.is_linked()
}
/// Iterate over allocated blocks.
///
/// Returns an iterator yielding [`AllocatedBlock`] values. Each [`AllocatedBlock`]
/// borrows `self` and is only valid for the duration of that borrow.
pub fn iter(&self) -> impl Iterator<Item = AllocatedBlock<'_>> + '_ {
let head = self.list.as_raw();
// SAFETY: Per the type invariant, `list` is an initialized sentinel `list_head`
// and is not concurrently modified (we hold a `&self` borrow). The list contains
// `gpu_buddy_block` items linked via `__bindgen_anon_1.link`. `Block` is
// `#[repr(transparent)]` over `gpu_buddy_block`.
let clist = unsafe {
clist_create!(
head,
Block,
bindings::gpu_buddy_block,
__bindgen_anon_1.link
)
};
clist
.iter()
.map(|this| AllocatedBlock { this, blocks: self })
}
}
#[pinned_drop]
impl PinnedDrop for AllocatedBlocks {
fn drop(self: Pin<&mut Self>) {
let guard = self.buddy.lock();
// SAFETY:
// - list is valid per the type's invariants.
// - guard provides exclusive access to the allocator.
unsafe {
bindings::gpu_buddy_free_list(guard.as_raw(), self.list.as_raw(), 0);
}
}
}
/// A GPU buddy block.
///
/// Transparent wrapper over C `gpu_buddy_block` structure. This type is returned
/// as references during iteration over [`AllocatedBlocks`].
///
/// # Invariants
///
/// The inner [`Opaque`] contains a valid, allocated `gpu_buddy_block`.
#[repr(transparent)]
struct Block(Opaque<bindings::gpu_buddy_block>);
impl Block {
/// Get a raw pointer to the underlying C block.
fn as_raw(&self) -> *mut bindings::gpu_buddy_block {
self.0.get()
}
/// Get the block's raw offset in the buddy address space (without base offset).
fn offset(&self) -> u64 {
// SAFETY: `self.as_raw()` is valid per the type's invariants.
unsafe { bindings::gpu_buddy_block_offset(self.as_raw()) }
}
/// Get the block order.
fn order(&self) -> u32 {
// SAFETY: `self.as_raw()` is valid per the type's invariants.
unsafe { bindings::gpu_buddy_block_order(self.as_raw()) }
}
}
// SAFETY: `Block` is a wrapper around `gpu_buddy_block` which can be
// sent across threads safely.
unsafe impl Send for Block {}
// SAFETY: `Block` is only accessed through shared references after
// allocation, and thus safe to access concurrently across threads.
unsafe impl Sync for Block {}
/// A buddy block paired with its owning [`AllocatedBlocks`] context.
///
/// Unlike a raw block, which only knows its offset within the buddy address
/// space, an [`AllocatedBlock`] also has access to the allocator's `base_offset`
/// and `chunk_size`, enabling it to compute absolute offsets and byte sizes.
///
/// Returned by [`AllocatedBlocks::iter()`].
pub struct AllocatedBlock<'a> {
this: &'a Block,
blocks: &'a AllocatedBlocks,
}
impl AllocatedBlock<'_> {
/// Get the block's offset in the address space.
///
/// Returns the absolute offset including the allocator's base offset.
/// This is the actual address to use for accessing the allocated memory.
pub fn offset(&self) -> u64 {
self.blocks.buddy.params.base_offset + self.this.offset()
}
/// Get the block order (size = chunk_size << order).
pub fn order(&self) -> u32 {
self.this.order()
}
/// Get the block's size in bytes.
pub fn size(&self) -> u64 {
(self.blocks.buddy.params.chunk_size.as_usize() as u64) << self.this.order()
}
}

9
rust/kernel/interop.rs Normal file
View file

@ -0,0 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
//! Infrastructure for interfacing Rust code with C kernel subsystems.
//!
//! This module is intended for low-level, unsafe Rust infrastructure code
//! that interoperates between Rust and C. It is *not* for use directly in
//! Rust drivers.
pub mod list;

339
rust/kernel/interop/list.rs Normal file
View file

@ -0,0 +1,339 @@
// SPDX-License-Identifier: GPL-2.0
//! Rust interface for C doubly circular intrusive linked lists.
//!
//! This module provides Rust abstractions for iterating over C `list_head`-based
//! linked lists. It should only be used for cases where C and Rust code share
//! direct access to the same linked list through a C interop interface.
//!
//! Note: This *must not* be used by Rust components that just need a linked list
//! primitive. Use [`kernel::list::List`] instead.
//!
//! # Examples
//!
//! ```
//! use kernel::{
//! bindings,
//! interop::list::clist_create,
//! types::Opaque,
//! };
//! # // Create test list with values (0, 10, 20) - normally done by C code but it is
//! # // emulated here for doctests using the C bindings.
//! # use core::mem::MaybeUninit;
//! #
//! # /// C struct with embedded `list_head` (typically will be allocated by C code).
//! # #[repr(C)]
//! # pub struct SampleItemC {
//! # pub value: i32,
//! # pub link: bindings::list_head,
//! # }
//! #
//! # let mut head = MaybeUninit::<bindings::list_head>::uninit();
//! #
//! # let head = head.as_mut_ptr();
//! # // SAFETY: `head` and all the items are test objects allocated in this scope.
//! # unsafe { bindings::INIT_LIST_HEAD(head) };
//! #
//! # let mut items = [
//! # MaybeUninit::<SampleItemC>::uninit(),
//! # MaybeUninit::<SampleItemC>::uninit(),
//! # MaybeUninit::<SampleItemC>::uninit(),
//! # ];
//! #
//! # for (i, item) in items.iter_mut().enumerate() {
//! # let ptr = item.as_mut_ptr();
//! # // SAFETY: `ptr` points to a valid `MaybeUninit<SampleItemC>`.
//! # unsafe { (*ptr).value = i as i32 * 10 };
//! # // SAFETY: `&raw mut` creates a pointer valid for `INIT_LIST_HEAD`.
//! # unsafe { bindings::INIT_LIST_HEAD(&raw mut (*ptr).link) };
//! # // SAFETY: `link` was just initialized and `head` is a valid list head.
//! # unsafe { bindings::list_add_tail(&mut (*ptr).link, head) };
//! # }
//!
//! /// Rust wrapper for the C struct.
//! ///
//! /// The list item struct in this example is defined in C code as:
//! ///
//! /// ```c
//! /// struct SampleItemC {
//! /// int value;
//! /// struct list_head link;
//! /// };
//! /// ```
//! #[repr(transparent)]
//! pub struct Item(Opaque<SampleItemC>);
//!
//! impl Item {
//! pub fn value(&self) -> i32 {
//! // SAFETY: `Item` has the same layout as `SampleItemC`.
//! unsafe { (*self.0.get()).value }
//! }
//! }
//!
//! // Create typed [`CList`] from sentinel head.
//! // SAFETY: `head` is valid and initialized, items are `SampleItemC` with
//! // embedded `link` field, and `Item` is `#[repr(transparent)]` over `SampleItemC`.
//! let list = unsafe { clist_create!(head, Item, SampleItemC, link) };
//!
//! // Iterate directly over typed items.
//! let mut found_0 = false;
//! let mut found_10 = false;
//! let mut found_20 = false;
//!
//! for item in list.iter() {
//! let val = item.value();
//! if val == 0 { found_0 = true; }
//! if val == 10 { found_10 = true; }
//! if val == 20 { found_20 = true; }
//! }
//!
//! assert!(found_0 && found_10 && found_20);
//! ```
use core::{
iter::FusedIterator,
marker::PhantomData, //
};
use crate::{
bindings,
types::Opaque, //
};
use pin_init::{
pin_data,
pin_init,
PinInit, //
};
/// FFI wrapper for a C `list_head` object used in intrusive linked lists.
///
/// # Invariants
///
/// - The underlying `list_head` is initialized with valid non-`NULL` `next`/`prev` pointers.
#[pin_data]
#[repr(transparent)]
pub struct CListHead {
#[pin]
inner: Opaque<bindings::list_head>,
}
impl CListHead {
/// Create a `&CListHead` reference from a raw `list_head` pointer.
///
/// # Safety
///
/// - `ptr` must be a valid pointer to an initialized `list_head` (e.g. via
/// `INIT_LIST_HEAD()`), with valid non-`NULL` `next`/`prev` pointers.
/// - `ptr` must remain valid for the lifetime `'a`.
/// - The list and all linked `list_head` nodes must not be modified from
/// anywhere for the lifetime `'a`, unless done so via any [`CListHead`] APIs.
#[inline]
pub unsafe fn from_raw<'a>(ptr: *mut bindings::list_head) -> &'a Self {
// SAFETY:
// - `CListHead` has the same layout as `list_head`.
// - `ptr` is valid and unmodified for `'a` per caller guarantees.
unsafe { &*ptr.cast() }
}
/// Get the raw `list_head` pointer.
#[inline]
pub fn as_raw(&self) -> *mut bindings::list_head {
self.inner.get()
}
/// Get the next [`CListHead`] in the list.
#[inline]
pub fn next(&self) -> &Self {
let raw = self.as_raw();
// SAFETY:
// - `self.as_raw()` is valid and initialized per type invariants.
// - The `next` pointer is valid and non-`NULL` per type invariants
// (initialized via `INIT_LIST_HEAD()` or equivalent).
unsafe { Self::from_raw((*raw).next) }
}
/// Check if this node is linked in a list (not isolated).
#[inline]
pub fn is_linked(&self) -> bool {
let raw = self.as_raw();
// SAFETY: `self.as_raw()` is valid per type invariants.
unsafe { (*raw).next != raw && (*raw).prev != raw }
}
/// Returns a pin-initializer for the list head.
pub fn new() -> impl PinInit<Self> {
pin_init!(Self {
// SAFETY: `INIT_LIST_HEAD` initializes `slot` to a valid empty list.
inner <- Opaque::ffi_init(|slot| unsafe { bindings::INIT_LIST_HEAD(slot) }),
})
}
}
// SAFETY: `list_head` contains no thread-bound state; it only holds
// `next`/`prev` pointers.
unsafe impl Send for CListHead {}
// SAFETY: `CListHead` can be shared among threads as modifications are
// not allowed at the moment.
unsafe impl Sync for CListHead {}
impl PartialEq for CListHead {
#[inline]
fn eq(&self, other: &Self) -> bool {
core::ptr::eq(self, other)
}
}
impl Eq for CListHead {}
/// Low-level iterator over `list_head` nodes.
///
/// An iterator used to iterate over a C intrusive linked list (`list_head`). The caller has to
/// perform conversion of returned [`CListHead`] to an item (using [`container_of`] or similar).
///
/// # Invariants
///
/// `current` and `sentinel` are valid references into an initialized linked list.
struct CListHeadIter<'a> {
/// Current position in the list.
current: &'a CListHead,
/// The sentinel head (used to detect end of iteration).
sentinel: &'a CListHead,
}
impl<'a> Iterator for CListHeadIter<'a> {
type Item = &'a CListHead;
#[inline]
fn next(&mut self) -> Option<Self::Item> {
// Check if we've reached the sentinel (end of list).
if self.current == self.sentinel {
return None;
}
let item = self.current;
self.current = item.next();
Some(item)
}
}
impl<'a> FusedIterator for CListHeadIter<'a> {}
/// A typed C linked list with a sentinel head intended for FFI use-cases where
/// a C subsystem manages a linked list that Rust code needs to read. Generally
/// required only for special cases.
///
/// A sentinel head [`CListHead`] represents the entire linked list and can be used
/// for iteration over items of type `T`; it is not associated with a specific item.
///
/// The const generic `OFFSET` specifies the byte offset of the `list_head` field within
/// the struct that `T` wraps.
///
/// # Invariants
///
/// - The sentinel [`CListHead`] has valid non-`NULL` `next`/`prev` pointers.
/// - `OFFSET` is the byte offset of the `list_head` field within the struct that `T` wraps.
/// - All the list's `list_head` nodes have valid non-`NULL` `next`/`prev` pointers.
#[repr(transparent)]
pub struct CList<T, const OFFSET: usize>(CListHead, PhantomData<T>);
impl<T, const OFFSET: usize> CList<T, OFFSET> {
/// Create a typed [`CList`] reference from a raw sentinel `list_head` pointer.
///
/// # Safety
///
/// - `ptr` must be a valid pointer to an initialized sentinel `list_head` (e.g. via
/// `INIT_LIST_HEAD()`), with valid non-`NULL` `next`/`prev` pointers.
/// - `ptr` must remain valid for the lifetime `'a`.
/// - The list and all linked nodes must not be concurrently modified for the lifetime `'a`.
/// - The list must contain items where the `list_head` field is at byte offset `OFFSET`.
/// - `T` must be `#[repr(transparent)]` over the C struct.
#[inline]
pub unsafe fn from_raw<'a>(ptr: *mut bindings::list_head) -> &'a Self {
// SAFETY:
// - `CList` has the same layout as `CListHead` due to `#[repr(transparent)]`.
// - Caller guarantees `ptr` is a valid, sentinel `list_head` object.
unsafe { &*ptr.cast() }
}
/// Check if the list is empty.
#[inline]
pub fn is_empty(&self) -> bool {
!self.0.is_linked()
}
/// Create an iterator over typed items.
#[inline]
pub fn iter(&self) -> CListIter<'_, T, OFFSET> {
let head = &self.0;
CListIter {
head_iter: CListHeadIter {
current: head.next(),
sentinel: head,
},
_phantom: PhantomData,
}
}
}
/// High-level iterator over typed list items.
pub struct CListIter<'a, T, const OFFSET: usize> {
head_iter: CListHeadIter<'a>,
_phantom: PhantomData<&'a T>,
}
impl<'a, T, const OFFSET: usize> Iterator for CListIter<'a, T, OFFSET> {
type Item = &'a T;
#[inline]
fn next(&mut self) -> Option<Self::Item> {
let head = self.head_iter.next()?;
// Convert to item using `OFFSET`.
//
// SAFETY: The pointer calculation is valid because `OFFSET` is derived
// from `offset_of!` per type invariants.
Some(unsafe { &*head.as_raw().byte_sub(OFFSET).cast::<T>() })
}
}
impl<'a, T, const OFFSET: usize> FusedIterator for CListIter<'a, T, OFFSET> {}
/// Create a C doubly-circular linked list interface [`CList`] from a raw `list_head` pointer.
///
/// This macro creates a `CList<T, OFFSET>` that can iterate over items of type `$rust_type`
/// linked via the `$field` field in the underlying C struct `$c_type`.
///
/// # Arguments
///
/// - `$head`: Raw pointer to the sentinel `list_head` object (`*mut bindings::list_head`).
/// - `$rust_type`: Each item's Rust wrapper type.
/// - `$c_type`: Each item's C struct type that contains the embedded `list_head`.
/// - `$field`: The name of the `list_head` field within the C struct.
///
/// # Safety
///
/// The caller must ensure:
///
/// - `$head` is a valid, initialized sentinel `list_head` (e.g. via `INIT_LIST_HEAD()`)
/// pointing to a list that is not concurrently modified for the lifetime of the [`CList`].
/// - The list contains items of type `$c_type` linked via an embedded `$field`.
/// - `$rust_type` is `#[repr(transparent)]` over `$c_type` or has compatible layout.
///
/// # Examples
///
/// Refer to the examples in the [`crate::interop::list`] module documentation.
#[macro_export]
macro_rules! clist_create {
($head:expr, $rust_type:ty, $c_type:ty, $($field:tt).+) => {{
// Compile-time check that field path is a `list_head`.
let _: fn(*const $c_type) -> *const $crate::bindings::list_head =
|p| &raw const (*p).$($field).+;
// Calculate offset and create `CList`.
const OFFSET: usize = ::core::mem::offset_of!($c_type, $($field).+);
$crate::interop::list::CList::<$rust_type, OFFSET>::from_raw($head)
}};
}
pub use clist_create;

3
rust/kernel/lib.rs
View file

@ -72,12 +72,15 @@ pub mod faux;
pub mod firmware;
pub mod fmt;
pub mod fs;
#[cfg(CONFIG_GPU_BUDDY = "y")]
pub mod gpu;
#[cfg(CONFIG_I2C = "y")]
pub mod i2c;
pub mod id_pool;
#[doc(hidden)]
pub mod impl_flags;
pub mod init;
pub mod interop;
pub mod io;
pub mod ioctl;
pub mod iommu;

91
rust/kernel/uaccess.rs
View file

@ -7,10 +7,12 @@
use crate::{
alloc::{Allocator, Flags},
bindings,
dma::Coherent,
error::Result,
ffi::{c_char, c_void},
fs::file,
prelude::*,
ptr::KnownSize,
transmute::{AsBytes, FromBytes},
};
use core::mem::{size_of, MaybeUninit};
@ -459,20 +461,19 @@ impl UserSliceWriter {
self.length == 0
}
/// Writes raw data to this user pointer from a kernel buffer.
/// Low-level write from a raw pointer.
///
/// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
/// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
/// if it returns an error.
pub fn write_slice(&mut self, data: &[u8]) -> Result {
let len = data.len();
let data_ptr = data.as_ptr().cast::<c_void>();
/// # Safety
///
/// The caller must ensure that `from` is valid for reads of `len` bytes.
unsafe fn write_raw(&mut self, from: *const u8, len: usize) -> Result {
if len > self.length {
return Err(EFAULT);
}
// SAFETY: `data_ptr` points into an immutable slice of length `len`, so we may read
// that many bytes from it.
let res = unsafe { bindings::copy_to_user(self.ptr.as_mut_ptr(), data_ptr, len) };
// SAFETY: Caller guarantees `from` is valid for `len` bytes (see this function's
// safety contract).
let res = unsafe { bindings::copy_to_user(self.ptr.as_mut_ptr(), from.cast(), len) };
if res != 0 {
return Err(EFAULT);
}
@ -481,6 +482,76 @@ impl UserSliceWriter {
Ok(())
}
/// Writes raw data to this user pointer from a kernel buffer.
///
/// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
/// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
/// if it returns an error.
pub fn write_slice(&mut self, data: &[u8]) -> Result {
// SAFETY: `data` is a valid slice, so `data.as_ptr()` is valid for
// reading `data.len()` bytes.
unsafe { self.write_raw(data.as_ptr(), data.len()) }
}
/// Writes raw data to this user pointer from a DMA coherent allocation.
///
/// Copies `count` bytes from `alloc` starting from `offset` into this userspace slice.
///
/// # Errors
///
/// - [`EOVERFLOW`]: `offset + count` overflows.
/// - [`ERANGE`]: `offset + count` exceeds the size of `alloc`, or `count` exceeds the
/// size of the user-space buffer.
/// - [`EFAULT`]: the write hits a bad address or goes out of bounds of this
/// [`UserSliceWriter`].
///
/// This call may modify the associated userspace slice even if it returns an error.
///
/// Note: The memory may be concurrently modified by hardware (e.g., DMA). In such cases,
/// the copied data may be inconsistent, but this does not cause undefined behavior.
///
/// # Example
///
/// Copy the first 256 bytes of a DMA coherent allocation into a userspace buffer:
///
/// ```no_run
/// use kernel::uaccess::UserSliceWriter;
/// use kernel::dma::Coherent;
///
/// fn copy_dma_to_user(
/// mut writer: UserSliceWriter,
/// alloc: &Coherent<[u8]>,
/// ) -> Result {
/// writer.write_dma(alloc, 0, 256)
/// }
/// ```
pub fn write_dma<T: KnownSize + AsBytes + ?Sized>(
&mut self,
alloc: &Coherent<T>,
offset: usize,
count: usize,
) -> Result {
let len = alloc.size();
if offset.checked_add(count).ok_or(EOVERFLOW)? > len {
return Err(ERANGE);
}
if count > self.len() {
return Err(ERANGE);
}
// SAFETY: `as_ptr()` returns a valid pointer to a memory region of `count()` bytes, as
// guaranteed by the `Coherent` invariants. The check above ensures `offset + count <= len`.
let src_ptr = unsafe { alloc.as_ptr().cast::<u8>().add(offset) };
// Note: Use `write_raw` instead of `write_slice` because the allocation is coherent
// memory that hardware may modify (e.g., DMA); we cannot form a `&[u8]` slice over
// such volatile memory.
//
// SAFETY: `src_ptr` points into the allocation and is valid for `count` bytes (see above).
unsafe { self.write_raw(src_ptr, count) }
}
/// Writes raw data to this user pointer from a kernel buffer partially.
///
/// This is the same as [`Self::write_slice`] but considers the given `offset` into `data` and

104
rust/kernel/workqueue.rs
View file

@ -189,12 +189,18 @@ use crate::{
alloc::{AllocError, Flags},
container_of,
prelude::*,
sync::Arc,
sync::LockClassKey,
sync::{
aref::{
ARef,
AlwaysRefCounted, //
},
Arc,
LockClassKey, //
},
time::Jiffies,
types::Opaque,
};
use core::marker::PhantomData;
use core::{marker::PhantomData, ptr::NonNull};
/// Creates a [`Work`] initialiser with the given name and a newly-created lock class.
#[macro_export]
@ -425,10 +431,11 @@ pub unsafe trait RawDelayedWorkItem<const ID: u64>: RawWorkItem<ID> {}
/// Defines the method that should be called directly when a work item is executed.
///
/// This trait is implemented by `Pin<KBox<T>>` and [`Arc<T>`], and is mainly intended to be
/// implemented for smart pointer types. For your own structs, you would implement [`WorkItem`]
/// instead. The [`run`] method on this trait will usually just perform the appropriate
/// `container_of` translation and then call into the [`run`][WorkItem::run] method from the
/// This trait is implemented by `Pin<KBox<T>>`, [`Arc<T>`] and [`ARef<T>`], and
/// is mainly intended to be implemented for smart pointer types. For your own
/// structs, you would implement [`WorkItem`] instead. The [`run`] method on
/// this trait will usually just perform the appropriate `container_of`
/// translation and then call into the [`run`][WorkItem::run] method from the
/// [`WorkItem`] trait.
///
/// This trait is used when the `work_struct` field is defined using the [`Work`] helper.
@ -934,6 +941,89 @@ where
{
}
// SAFETY: Like the `Arc<T>` implementation, the `__enqueue` implementation for
// `ARef<T>` obtains a `work_struct` from the `Work` field using
// `T::raw_get_work`, so the same safety reasoning applies:
//
// - `__enqueue` gets the `work_struct` from the `Work` field, using `T::raw_get_work`.
// - The only safe way to create a `Work` object is through `Work::new`.
// - `Work::new` makes sure that `T::Pointer::run` is passed to `init_work_with_key`.
// - Finally `Work` and `RawWorkItem` guarantee that the correct `Work` field
// will be used because of the ID const generic bound. This makes sure that `T::raw_get_work`
// uses the correct offset for the `Work` field, and `Work::new` picks the correct
// implementation of `WorkItemPointer` for `ARef<T>`.
unsafe impl<T, const ID: u64> WorkItemPointer<ID> for ARef<T>
where
T: AlwaysRefCounted,
T: WorkItem<ID, Pointer = Self>,
T: HasWork<T, ID>,
{
unsafe extern "C" fn run(ptr: *mut bindings::work_struct) {
// The `__enqueue` method always uses a `work_struct` stored in a `Work<T, ID>`.
let ptr = ptr.cast::<Work<T, ID>>();
// SAFETY: This computes the pointer that `__enqueue` got from
// `ARef::into_raw`.
let ptr = unsafe { T::work_container_of(ptr) };
// SAFETY: The safety contract of `work_container_of` ensures that it
// returns a valid non-null pointer.
let ptr = unsafe { NonNull::new_unchecked(ptr) };
// SAFETY: This pointer comes from `ARef::into_raw` and we've been given
// back ownership.
let aref = unsafe { ARef::from_raw(ptr) };
T::run(aref)
}
}
// SAFETY: The `work_struct` raw pointer is guaranteed to be valid for the duration of the call to
// the closure because we get it from an `ARef`, which means that the ref count will be at least 1,
// and we don't drop the `ARef` ourselves. If `queue_work_on` returns true, it is further guaranteed
// to be valid until a call to the function pointer in `work_struct` because we leak the memory it
// points to, and only reclaim it if the closure returns false, or in `WorkItemPointer::run`, which
// is what the function pointer in the `work_struct` must be pointing to, according to the safety
// requirements of `WorkItemPointer`.
unsafe impl<T, const ID: u64> RawWorkItem<ID> for ARef<T>
where
T: AlwaysRefCounted,
T: WorkItem<ID, Pointer = Self>,
T: HasWork<T, ID>,
{
type EnqueueOutput = Result<(), Self>;
unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput
where
F: FnOnce(*mut bindings::work_struct) -> bool,
{
let ptr = ARef::into_raw(self);
// SAFETY: Pointers from ARef::into_raw are valid and non-null.
let work_ptr = unsafe { T::raw_get_work(ptr.as_ptr()) };
// SAFETY: `raw_get_work` returns a pointer to a valid value.
let work_ptr = unsafe { Work::raw_get(work_ptr) };
if queue_work_on(work_ptr) {
Ok(())
} else {
// SAFETY: The work queue has not taken ownership of the pointer.
Err(unsafe { ARef::from_raw(ptr) })
}
}
}
// SAFETY: By the safety requirements of `HasDelayedWork`, the `work_struct` returned by methods in
// `HasWork` provides a `work_struct` that is the `work` field of a `delayed_work`, and the rest of
// the `delayed_work` has the same access rules as its `work` field.
unsafe impl<T, const ID: u64> RawDelayedWorkItem<ID> for ARef<T>
where
T: WorkItem<ID, Pointer = Self>,
T: HasDelayedWork<T, ID>,
T: AlwaysRefCounted,
{
}
/// Returns the system work queue (`system_wq`).
///
/// It is the one used by `schedule[_delayed]_work[_on]()`. Multi-CPU multi-threaded. There are