Interactive Art Opens a Door into Virtuality: Intel® Software Innovator Ke Jyun Wu Explains How

ID 750712
Updated 12/19/2021
Version Latest



Ke Jyun Wu develops interactive visual art to bring the virtual work into reality.

Ke Jyun Wu is a generative artist and programmer based in Taiwan. He combines his expertise in technology and sensitivity to the visual arts to explore the possibilities of generative art and collaboration.

Wu’s challenge is to redefine the boundary between reality and virtuality through sound-visualizing performances, public art installations, and digital choreography performances – with exhibits in the U.S., South Korea, Russia, China, and Taiwan.

In order to break the barrier between the virtual world and reality, he takes inspiration from a wide range of landscapes, both real and imagined. “In the future we may experience life like in the movie, Ready Player One,” said Wu. “At that point, our lives may be built on the virtual world. What I’m working on now is to explore how natural landscapes can be transformed into virtual ones.”

Wu is also an active contributor to the Intel® Software Innovator Program, Intel’s community for forward-thinking developers. Participants share thought leadership and technology expertise as well as inspire their peers by speaking and giving demos of their work at industry events. Now expanding to support graphics developers and creators, the Intel Software Innovator Program offers an Xe Community track with an emphasis on applications and experiences demanding greater speed and performance even as workloads increase.

Video: Forest, part of Wu’s DigiScape series on distinguishing between real and virtual worlds.

Technology in Service of Imagination

To bring forward Wu’s vision, he uses MIDI (Musical Instrument Digital Interface) and the Unity* game engine to create his interactive DigiScapes, or digital landscapes. “MIDI control allows me to use gesture and movement to influence an interactive medium, including audiovisual performances,” said Wu.

This enables him to create a connection between reality and virtuality. “When I conceive the interaction that occurs in my DigiScapes, I think about the movement of light and the intensity of visual effects. I then develop the means to produce what’s in my imagination through the use of MIDI control and the software in Unity’s game engine, said Wu”.

Video: Watch how gestures affect the imagery in Wu’s Seabed DigiScape.

To build his imagery, Wu frequently uses Unity's Visual Effect Graph and Shader Graph. Compared with traditional visual coding, Unity’s node-based tools dramatically increase the speed of development. The short video below demonstrates how he handles depth-based blurring inside a sphere.

Video: See how Wu uses a series of layers in Unity 3D to produce sophisticated imagery.

Adding New Technologies to his Palette

To enhance the way Wu realizes his imagination through his work, he plans to incorporate additional innovative technologies into his processes. “I’ve so far focused on software and the Unity game engine's application., The chance to add external technologies will open up new, exciting possibilities,” said Wu.

Wu explained that he looks forward to using the Intel® Distribution of OpenVINO™ toolkit. “Applying artificial intelligence (AI) is what I long for, and I’ll look into how I can best implement AI in Unity,” Wu added.

The Process: from Ideation to Completion

According to Wu, he often dedicates an entire month to ideation before he begins any actual programming. “I must look at a large number of references in order to feed my imagination,” said Wu. Wu refers to this period as pre-construction. “Once that phase is complete, I’ll start technical testing and production,” Wu added.

He prefers to work with existing functions and technologies to rapidly complete his prototypes, then modifies or rewrites functions and screens according to his needs. The software primarily used is Unity 3D and the High Definition Render Pipeline (HDRP), a high-fidelity Scriptable Render Pipeline built by Unity for modern platforms that are compute shader compatible.

On Joining the Intel® Software Innovator Program

While Wu followed Intel® since he began working in technology, he only recently became a member of the Intel® Software Innovator Program and community.

“I’ve always been an end-user, and rarely had access to core technicians. After listening to my first NDA call, I am so excited that I can step out of my comfort zone and receive information shared by experts from different fields,” said Wu.

Current Projects Update

Wu continues to work on his DigiScape series, which he views as a never-ending project. He can reinterpret the scenery in all dimensions and integrate all kinds of
interaction to enable the audience to communicate with the creation and become a part of the experience.

However, he’s now incorporating augmented reality (AR) and photogrammetry as well as improving real-time calculation of image quality.

Intel is sponsoring Wu’s current project to show optimization for Intel® Arc Graphics solution, code named Alchemist. “In my new project, I'm using ray tracing-related technologies to improve the details and image quality,” said Wu. “And simultaneously, I’m working with 2D fluid simulation in a novel way to create 3D effects.”

Video: Watch a highlight of Wu’s interactive installations and motion work up to 2019, all made using Unity*.

Elevate Your Graphics Skills

Ignite inspiration, share achievements, network with peers and get access to experts by becoming an Intel® Software Innovator. Game devs, media developers and other creators can apply to the new Xe Community track – opening a world of opportunity to craft amazing applications, games and experiences running on Xe architecture.

To start connecting and sharing your projects, join the global developer community on Intel® DevMesh.