All the changes in Metal: Performance HUD, shader assistance, more

For two consecutive years at WWDC, Apple improved Metal, with Metal 3 running on Apple Silicon Macs and on iOS and tvOS devices. Here are the big changes to Metal in the forthcoming iOS 17 and macOS Sonoma.

Metal 3 represents significant enhancements to the Metal engine and frameworks for writing 3D games for Apple platforms.

Metal Shading Language Specification 1.3

Metal has its own C++14-based shading language, Metal Shading Language, which is fully detailed in the Metal Shading Language Specification (MSL).

At WWDC '23 Apple introduced version 1.3 of the Metal Shading Language Specification. The MSL spec covers every aspect of writing shader code for Metal in C++.

The new MSL specification can be downloaded as a PDF from Apple's developer website.

Metal Performance HUD

Metal 3 now includes an optional small Heads-Up Display visible in the upper right corner of rendered scenes as your game runs:

The Metal performance heads-up display.

The Performance HUD can display several pieces of useful information for inspecting the performance of your game in real time:

1. Show and log key performance statistics
2. CPU and resolution
3. Device refresh rate
4. Scaling
5. Direct or composited rendering
6. FPS average
7. GPU cost per frame
8. More

Some Metal heads-up display stats.

Metal Shader Converter tool

At WWDC '23 Apple also introduced the Metal Shader Converter tool. Shaders are small programs that manage light, color, and materials on 3D objects in a scene.

Metal Shader Converter helps you convert shaders written in other languages and frameworks or from other platforms to Metal 3 - namely from Microsoft Windows' DirectX 12 DXIL format.

Metal Shader Converter provides two binding models to choose from: Automatic Layout and Explicit Layout. These models help convert shaders from other platforms.

MTLBuffer tables are used in the Metal Shader Converter tool.

Bindless Rendering, argument buffers, and ray tracing

New in Metal 3, bindless rendering allows you to use ray tracing to improve performance and quality in 3D rendered scenes.

One aspect of bindless rendering - argument buffers - allow you to combine resources into buffers which speed up rendering.

Instead of binding each display resource, with argument buffers resources are linked together in memory in a buffer. In the bindless model, argument buffers aggregate and link together resources allowing direct access from rendering pipelines.

3D meshes are linked together in memory instead of being used one by one in rendering pipelines. This speeds up access to all the objects needed by ray tracing shaders to render high-resolution surfaces faster and with more detail.

Bindless Rendering.

Meshes, materials, and textures can all be linked in memory.

Four new enhancements in Metal 3 help bindless rendering:

Argument buffers - link resources together for direct access as described above.

Unbounded arrays - allows the definition of multiple Mesh structures in an array at once. Shaders can access arrays of any size without constraints.

Mesh struct objects in unbounded arrays can now be accessed directly from C/C++ code and threads, although you must use care when accessing them due to thread synchronization issues.

You can use the __METAL_VERSION__ precompiler flag to conditionally compile C, C++, and Mesh declarations in Metal 3 to use unbounded arrays, if available on the target platform.

You can use the MTLDevice object at runtime to see if any given device supports the new features. To use the new features you must have a device with the following specifications:

iOS - A13 Bionic chip or newer.

macOS - A 2016 Mac or better.

The "Buffers" section of Apple's Metal developer documentation has a new section on argument buffers.

MTLHeap-allocated acceleration structures

Heaps are pre-allocated memory spaces into which resources and structures are loaded prior to rendering. Being heap-resident means resources are preloaded and ready to use instantly when required during rendering.

Ray-tracing acceleration structures can be allocated directly from MTLHeap objects along with buffers and textures. This allows aggregation with other resource types. Using MTLHeap acceleration structures saves CPU time in threads.

Shader validation enhancements

Typically, resources must be heap-resident before rendering to ensure rendering performance and accurate display. Failure to heap-allocate resources in advance can lead to performance glitches and even incorrect object display during drawing.

To help ensure resources are heap-resident, Metal 3 provides a new feature to detect missing residency during shared command buffer execution. To ensure heap residency, useResource: is now sent to each element in a scene before the scene objects are sent to the ray-tracing encoder.

This form of insuring resources are resident before ray-tracing shaders run is called shader validation in Metal 3.

These four features make it easier to go bindless in Metal 3.

Also, be sure to check out the WWDC '23 sessions "Your guide to Metal ray tracing" and "Go bindless with Metal 3".

Metal debugger dependency viewer

The new Metal Debugger Dependency Viewer displays dependencies between render workload resources.

To display the Dependency Viewer, open the Metal Debugger and click the Dependencies row in the navigator on the left:

Opening the Metal Dependency Debugger.

A dependency graph will appear which shows dependencies in each render command. Clicking any item in a dependency graph shows details for that item in the new sidebar on the right of the Dependency Viewer.

There are two types of dependencies displayed in the Metal Debugger Dependency Viewer: data flow and synchronization. The solid lines in the Dependency Viewer show data flow, and dotted lines show dependencies that show synchronization between passes.

Metal Dependency Debugger graph.

Shader debugger

There is also a Shader Debugger built into Xcode 14 and later which allows you to perform pixel-level debugging of your shader code. You can select any rendered pixel displayed in any scene, click the Shader Debugger button in the lower right corner of the debugger window and it will jump to the exact spot in your shader code used to draw the object.

Miscellaneous enhancements

The Metal team now has four additional recommendations for optimizing Metal rendering pipelines:

1. Move copies before rendering.
2. Group commands of the same type.
3. Avoid empty encoders.
4. Optimize using MTLLoadAction and MTLStoreAction

The last recommendation — MTLLoadAction and MTLStoreAction — are used on attachments, actions to be executed before a rendering pass.

These recommendations are also covered in more detail in the WWDC '23 session "Go bindless with Metal 3".See the Apple Developer app for access to all the WWDC '23 Metal session videos.

Also be sure to check out MetalFX upscaling, which we covered in another article last week.

These improvements in Metal 3 mean Metal is better than ever as a platform for your 3D games and apps on Apple devices.