New Devices Offer Developers a Game-Friendly Form Factor to Expand Their User Base
Intel is working with a variety of laptop manufacturers to bring a striking new form factor to market: the companion display laptop. This innovative design provides up to a 50% increase in screen real estate vs. a standard laptop, which allows ample space for PC gamers to easily relocate their chat app or internet search window and shift other common multitasking functions to the companion display. More significantly, from a game developer perspective, the companion display is also an ideal location to offload screen elements such as maps, inventory, stats and other overlays, which might otherwise obscure the game.
The companion display concept is extremely appealing to users; a research study conducted by Intel in late 2018 reveals gamers are enthusiastic about where this class of device is headed. They are particularly interested in the ability to have a more immersive gaming experience by offloading any functions that stand between them and their view of the game.
“Why didn’t someone think about this years ago?”
— User research participant, 2018
If the appeal of the new device is not enough to persuade you to think about offloading some of your user interface (UI) elements to the second screen, consider this: recent data from one of Intel’s partners suggests that up to 40% of PC gamers already use a second monitor for gaming and productivity. With effective planning, most games can be reconfigured to run on both companion display laptops and existing multi-monitor setups, since they both take advantage of extended displays. Game developers who incorporate support for dual monitor functionality now can delight a large and growing population of existing users, while simultaneously designing for a new generation of laptops.
Laptop manufacturers are currently experimenting with a variety of screen sizes and layouts, as well as different hinge configurations. The first new systems are expected to be announced in 2019, with more advances to follow.
This technical paper covers the benefits of companion displays, software engineering considerations and case studies of three different development environments.
Common companion display configurations: hinged screen (left), and screen above keyboard (right)
Companion Displays Are the Future
A standard laptop has a single screen facing the user, and a keyboard and trackpad on the base. Now, some manufacturers are moving the keyboard closer to the user, abandoning the trackpad, and adding a half-height, full-width screen above the keyboard. Others envision adding a hinge to allow angling the companion display and multiple manufacturers are considering adding smaller, phone-size screens in various locations.
Multitasking while playing games on a standard, single screen laptop often requires keystrokes or alt-tab to move through programs. This breaks immersion and decreases focus. Additionally, open application icons and windows clutter the screen and distract from play.
Companion display laptops solve these problems by offering three key features: more screen real estate, better user focus, and better ergonomics.
More Screen Real Estate
A companion display allows moving functions, controls or menus that normally take up primary screen space or obscure the game. Users can offload game items such as inventory, maps, player stats and competitor information. Intel’s user research revealed that the two items gamers consistently discussed offloading were maps and inventory. Most participants saw putting inventory down on the companion display as a significant advantage because these items are often hidden from the screen during game play.
Better User Focus
When located directly below the primary screen, the companion display can also function as an extension of the primary screen, which helps maintain focus and enhance immersion. A companion display laptop may also provide a more focused experience than a dual monitor experience because players don’t need to move their head left or right. Instead they can stay focused on the laptop’s primary screen and occasionally glance down at the companion display when new information is required.
Better Ergonomics
Users sometimes struggle with multiple monitors. Swiveling from left to right can strain the eyes, neck and shoulders. Simply glancing up or down is ergonomically preferable to moving the head.
Several laptop manufacturers are experimenting with configurations that include either a companion display that is slightly angled toward the user in a fixed position or a companion display with a hinge that allows the user to determine the optimal angle. Both solutions enhance ergonomics and reduce muscle fatigue.
Software Engineering Considerations
A variety of companion display sizes may be integrated into laptops, from six-inch phone-type displays to full-width, half-height screens. Users may also have one or more external monitors plugged into their laptops to leverage even more screen real estate. Application developers wishing to take advantage of the companion display should follow good multi-display coding practices and application layout. For example, when developing applications that may use companion displays, the application should do the following:
- Query the existence of a companion display
- Query the resolution, aspect ratio and relative position/ orientation of each screen as it is configured in the Windows* Display Settings
- Provide the user with an option to use the application across multiple screens
- And lastly, if the user has chosen to use multiple screens, the application should place windows/elements on the primary and companion displays, scaled appropriately. The app should scale elements on the companion display for touch if any touch-interaction is expected for those elements. The application should also allow the user to turn the companion display on or off
To ensure their software takes full advantage of the new devices, developers should also consider how to leverage touch, hardware settings, and system performance on these new laptops as they would on any platform.
The companion display is also an ideal location to offload screen elements, such as maps, inventory, stats and other overlays, which might otherwise obscure the game.
Touch
Intel’s research shows that users expect the primary and companion displays to work independently. They want to use the keyboard and mouse to drive the primary display as they do today, but the location of the companion display above the keyboard makes touch an appealing input option. While many users would like the companion display to be a touch-enabled screen they also expect UI elements on the companion display to be scaled for touch.
Additionally, users want to move the cursor seamlessly between the upper and lower screens, with an up or down motion, rather than side to side. This should be the default position of the companion display relative to the primary screen. However, users may want to change the relative position of the companion display to the primary screen, so application developers should not assume that the above/below positioning is always the case. This means that application windows or UI elements should not cross a screen boundary (for example, a single window should not be resized to span both primary and companion displays; rather, two separate application windows should be created and displayed on each screen appropriately).
While laptop manufacturers may implement touch on the companion display, it is up to software vendors and application developers to decide whether they want to enhance their game with those touch capabilities. Some developers may create interesting new touch-based features for companion displays, while others may simply offload inventory and maps in a read-only state. Developers may also offload features to the companion display that allow users to interact with those features using a keyboard and/or mouse. Numerous options may be explored.
Hardware Settings
On initial designs, the companion display screen will be 50% smaller than the upper, primary screen. Aspect ratios, screen resolutions, and the physical size and location will vary by manufacturer. Developers should ensure they identify the upper screen as the “main display” in Windows® 10 display settings.
Keep in mind that the two screens may use different settings, and that these are user-configurable. The software must be able to react to objects or application windows being moved by users across screens with different configurations.
Differences in frame rate between the primary and companion display, and auto-sizing for the companion display, may also require fine-tuning by developers.
Performance
Flexible software designs will have to handle performance differences between primary and companion display seamlessly. Features such as touch, gesture mapping and pen points should be taken into account.
Case Studies Demonstrate Ease of Development
Intel reached out to several development teams to gauge how their applications ported to a companion display configuration. Depending on the application type and development environment, they used different mechanisms to develop for companion displays. However, across the board, they found that following established conventions and interfaces provided by development tools and operating systems made modifications required for companion display quite straightforward.
Win32 Applications (C/C++/C#)
One game developer used two monitors to enhance display and control of an existing Win32-based racing game. Both primary and companion displays were left in landscape mode with the primary display at 1920x1080 resolution and the companion display at 1920x720. The primary screen was tested at multiple scales, with 125% proving optimal; the companion display remained at 100% scaling.
Win32-based racing game on different companion display prototypes
UWP streaming application on different companion display prototypes
The developers reported no perceivable performance differences between the two while running the game. However, matching the brightness and display characteristics required fine-tuning. As a result of the case study, the developer has decided to implement multiscreen support in their production game, handling any companion or extended display using these design mechanisms.
Universal Windows* Platform (UWP)
The second test involved a streaming web application using UWP. The development team found that using multiple cascading style sheets (CSS) did not require predefining screen resolutions, ratios, and positions for the companion display.
With their architecture, the UI on the companion display was created entirely with HTML/Javascript*. To put previews on the second screen, their engine provided that screen’s web UI extension window with WebGL* textures. The team reported that they could rebuild their application to use companion displays with minimal changes to their code.
In addition to using CSS, UWP application development for multiple displays can be accomplished using existing practices and UWP mechanisms provided by Microsoft*.
Good responsive design and use of these mechanisms can automate the application’s response to various screen positions, sizes, and scaling. Two helpful UWP code samples are provided by Microsoft:
- Projection sample: How to display two different views of your UWP app UI on different screens.
- Multiple Views sample: How to create and display multiple windows or views of your UWP app, specifying the default view, and switching between views.
Unity* Software
The third project used Unity* software. It used well-documented technology that allowed up to eight views of a game to be displayed on up to eight monitors simultaneously. This offers potential advantages for PC games, arcade games, and public installations.
As explained in their manuals, the default display is one monitor, so developers must explicitly activate additional displays via scripting, using Display.Activate. Once activated, they cannot be deactivated.
The best time to activate additional displays is on creating a new Scene. A good way to do this is to attach a script component to the default camera. Make sure you call Display.Activate only once during the startup.
You may find it helpful to create a small initial scene to test additional displays. Game view can be used to evaluate the look of the primary and companion displays.
Get Started Today with Companion Displays
This new breed of laptop offers interesting possibilities for developers to create unique gaming experiences. A companion display enables you to remove distractions from the primary screen, so gamers can focus on the game and developers can add information that helps them. When designed right, any changes that are made for the companion display will work seamlessly on a dual monitor setup as well.
Updating PC games to take advantage of companion displays can be very straightforward regardless of your development environment. The key is to first query for the existence of a second screen, then query the resolution, aspect ratio and the screen position. This will allow you to scale appropriately for the companion display.
Multiple laptop manufacturers are pursuing companion displays and we expect these devices to be available in the market beginning in 2019. Intel is looking for software partners to begin building out the ecosystem for companion display laptops.
For further information contact Intel Senior Account Manager Bryan Hobbs at bryan.hobbs@intel.com.
Learn More
Get started today learning how to develop for companion displays and drive new user experiences. For more information about guidance, application programming interfaces (APIs), and examples for multiscreen development of Win32 applications, visit the Windows Dev Center*.
Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
Microsoft and Windows are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries.
*Other names and brands may be claimed as the property of others. © 2019 Intel Corporation