Inside GM’s virtual assembly line where AI and VR revolutionize car manufacturing
By Jackie Charniga, Detroit Free Press
Solving manufacturing problems at a global automotive company used to require a lot more travel.
But General Motors employees today solve problems that require hands-on methods in the same room ― even on opposite ends of the country.
More effective than a Zoom screen and cheaper than airplane travel, virtual vehicle construction is just one way that virtual reality technology has disrupted GM’s entire pre-production process. And now, with the addition of artificial intelligence, the process of using virtual spaces to solve real-world manufacturing problems has gotten even more sophisticated.
GM workers across the company log in to virtual plants ― realistically mapped down to the microwave ― and work side by side on vehicles years before a prototype is ever constructed in real life.
“When we do a build for Spring Hill (Tennessee), it’s run here. They don’t have to set up any of the simulations ― they have the same equipment. They can drop in and be there with us,” Max Sikorski told the Free Press, part of the USA TODAY Network. “They see each other, talk to each other ― a real application of a metaverse.”
Sikorski leads the virtual reality team at GM’s Virtual Reality Assembly Center, one of the largest VR labs in the world housed in a 10,000-square-foot space at GM’s Global Artisan Innovation Center in Warren, Michigan. The various Global Artisan teams are responsible for GM’s prototype vehicles, low production vehicles, special editions and repair work.
The VR program launched in 2022, but starting this year, every single future GM vehicle will undergo the VR testing process.
So far, there have been six major build events this year that bring manufacturing employees from across GM’s operations to the virtual world to develop techniques and manage kinks on pre-pre-production vehicles.
The goal? Fine-tune product development and manufacturing processes years in advance to identify issues in the short term without building physical vehicle prototypes, conducting alterations to already constructed manufacturing operations or risking workplace injury ― steps that save the company considerable time and money.
“We found 494 issues this year without ever touching a real vehicle,” Sikorski said. “Issues in our world are a good thing — because if we catch an issue now, it’s not going to be an issue later.”
During a recent visit, large black dividers bisect the large warehouse, shielding all but one cleared area where a studio-like platform surrounded by lights and cords comprise the virtual reality testing space.
“It’s mostly closed off because you’re not allowed to see most of the stuff that goes on it here,” Sikorski said. “We work on everything several years before it hits even real prototypes.”
Sikorski confirmed details behind the curtain, immediately in front of the stage, concealed vehicles slated for 2028 production.
“This isn’t theoretical. GM isn’t piloting this ― GM is doing this, in every single vehicle program across its lineup,” Sikorski said.
Virtual reality timeline
The Detroit Three have experimented with virtual reality for well over a decade, updating these processes as the technology advances. Adding artificial intelligence capabilities to pre-production simulations is a recent change that allows even greater returns on GM’s progress.
Starting this year, his team of about a dozen engineers and manufacturing operators will virtually assemble every single future GM vehicle, all the while measuring the impact on the body of a seasoned employee.
Dwaine Carswell, a virtual assembly inspector, is one such expert. A UAW-represented worker, Carswell meets the automaker-imposed requirement having over 15 years assembly experience at a GM manufacturing plant.
As he moves, sensors strapped across his body with the tracking system light up on the screen where a virtual duplicate is projected. If the body part he’s using lights up red, that means the system believes if workers like Carswell continue moving that way on the line, it could cause strain or an injury.
That data comes from the U.S. Car Ergonomic Model that Sikorski said the Detroit Three use. The model operates on two standards ― information on the 95th percentile male body and 50th percentile female. If the job is deemed safe for both models, it means the vast majority of potential workers can handle those movements across an eight-hour shift.
“It’s beneficial for GM, because we want people to be able to swap in and out of jobs, but it’s also beneficial for the people because if you are a smaller person or a bigger person, you’re not going to be asked to do a job that wasn’t validated for your size,” Sikorski said.
GM currently uses Xsens suits, which results in Carswell’s digital duplicate to appear as a partially deconstructed department store mannequin, but Sikorski said GM is looking into switching to a similar motion capture suit used in moviemaking.
Since initiating the program, Sikorski’s team tracked and resolved over 50 ergonomic-related safety issues.
“A lot of companies do VR simulation, but they wait for an issue to come up,” he said. “We do exploratory VR, which means we build the entirety of the vehicle, everything a human touches, to get feedback years in advance. It’s an incredible time and money investment to do that, but we find it’s really valuable on the other end.”
Moving towards the virtual reality station during the Sept. 8 visit, Karsten Garbe, the plant director for global pre-production operations at GM, gestured broadly at one of the massive rooms.
“This used to be filled with workers building prototypes,” he said. “Now, as you can see, it is almost empty.”
Well, not entirely empty.
Building the team
Garbe estimates that 15 employees previously engaged in prototype construction were moved to other departments. Because of their high level of technical experience, he said it was easy for them to transfer to other departments, and transitioning away from physical fabrication has been a boon for the automaker. According to GM, it can cost up to $100,000 to build one prototype vehicle.
GM employs 700 skilled trade workers at the Artisan Center all requiring a four-year apprenticeship or other high-level certification. Hiring directly from dealership service departments poses another workaround, whereas it would take those employees about a year to reach the aptitude needed for intricate vehicle construction.
This is what makes them ideal candidates for the company’s virtual reality program.
“If we need, for example, some of them in another job, they are fully flexible. They are, from an educational point of view, usable in all areas of manufacturing and engineering in our facility,” Garbe said. “These gentlemen absolutely can do a lot of things artificial intelligence cannot do.”
Most of Sikorski’s staff came from within GM, Garbe said, while a few were volunteers that answered the call during COVID-19 when the automaker pivoted to production of personal protection equipment.
Garbe is also responsible for pre-production operations in South Korea, Brazil, and the joint venture in China, overseeing roughly 2,000 hourly and salaried workers.
While there were many volunteers ― particularly among employees eager to exchange working on the line for playing with virtual reality ― not everyone is suited for the task.
“Not everyone handles VR headsets well,” Sikorski said. “We don’t keep them in the headset for more than 2 hours at a time.”
Besides motion sickness, some workers lose control under VR helmets, either losing hearing outside sounds coming from within the helmet or extending arms involuntarily.
In addition to Carswell, four other operators work in the headsets for VR testing.
Outside manufacturing team members, Sikorski’s crew is composed of mechanical engineers, bioengineers and computer programmers. Problem-solving mindsets and curiosity in the process are the main requirements.
“Nobody teaches this stuff at schools. The software is too expensive, or the equipment is too expensive. We just hunt down people on staff at GM,” Sikorski said. “We can teach you about vehicles, but we can’t teach you to be interested in the technology.”
Where it starts
Before any vehicle is destined for mass production, GM constructed hundreds of prototypes to ensure construction plans would translate to the assembly line process, Sikorski said. For decades, that trial and error process required legions of workers and innumerable supplies of materials and parts.
“Not the most cost-effective way to do it, but it was the only way we had,” he said. “Finally, in the late 2020s, virtual reality caught up to where we could start putting people in VR to build the vehicles as if they’re real, with physics, in a virtual environment.”
By “physics” Sikorski means vehicle components respond in the virtual world the same way they would do in reality.
Objects do not pass through one another in the digital assembly plant, and workers simulating work have to actually go through the motions needed to affix a part inside the virtual vehicle. The tools workers use mirror exactly what the worker would use in the plant, and the vehicle they “see” inside virtual reality is the same height it would be on the line. When the part slides into place, the user feels a haptic buzz.
Because of the time constraints — workers get about 45 seconds to complete each task — a part that requires 14 motions to install instead of two or three opens the door to potential mistakes on the line.
That’s where Carswell comes in — even if the computer posits a movement is technically possible, his vast assembly experience allows him to push back on what may be unreasonable in person.
Virtual reality begins with a computer aided design, a 3D model GM has used since the 1990s that provides accurate dimensions and angles. This previously inspired material prototypes, but now offers data for the programmers to apply to their virtual system.
“That way, we’re not spending any money on real stuff until they have the design finished on their computer,” Sikorski said.
There’s two levels of virtuality that GM uses in the VRAC ― virtual reality, where the user is completely blind to the outside world via a Microsoft-produced headset, and augmented reality, where users can see the room through clear glass through the HTC Vive headset while interacting with computer-generated 3D images. Augmented reality is limited to the room where it is used, but since the entire space in virtual reality is fabricated, the possibilities are infinite.
Still, even if the team manages to find the most ideal building circumstances, that doesn’t mean the process is complete. Constructed vehicles must also be repairable, and Sikorski’s team increasingly works with GM’s internal service experts to ensure that they can speak up on areas they could see being challenging to fix.
“You might stack things differently when you’re building it, but that might make things difficult to take out,” Sikorski said. “Service would traditionally have to wait until we had a finished design. But they can now help us design our vehicles to be more serviceable so they don’t have to spend 16 hours on what should be a two-hour job.”
AI applications
The team is currently working on employing artificial intelligence to optimize the build process by tapping into the company’s century of carbuilding data to ensure time isn’t wasted solving problems that engineers tackled decades ago.
Likewise, if the virtual reality team encounters a problem that could add cost to manufacturing or strain on an employee, the AI data set could allow them to quickly determine any similar events GM recorded in the past of that same concern and what was done about them.
“It’s impossible for any one engineer to know what happened 45 years ago. It’s easy for an AI to know that,” he said. “That way, we don’t have to engineer it or do a solution that wasn’t as good. It’s not about getting rid of people; It’s about making their job easier, their time more efficient. I’m not going to waste X number of days on solving a problem we’ve already solved.”
Jackie Charniga covers General Motors for the Free Press. Reach her at jcharniga@freepress.com.
This article originally appeared on Detroit Free Press: Inside GM’s virtual assembly line where AI and VR revolutionize car manufacturing
USA TODAY Network via Reuters Connect