The speed with which latest technologies hit the market is nothing in comparison with the speed with which talented researchers find creative ways to make use of them, train them, even turn them into things we are able to’t live without. One such researcher is MIT MAD Fellow Alexander Htet Kyaw, a graduate student pursuing dual master’s degrees in architectural studies in computation and in electrical engineering and computer science.
Kyaw takes technologies like artificial intelligence, augmented reality, and robotics, and combines them with gesture, speech, and object recognition to create human-AI workflows which have the potential to interact with our built environment, change how we shop, design complex structures, and make physical things.
Considered one of his latest innovations is Curator AI, for which he and his MIT graduate student partners took first prize — $26,000 in OpenAI products and money — on the MIT AI Conference’s AI Construct: Generative Voice AI Solutions, a weeklong hackathon at MIT with final presentations held last fall in Latest York City. Working with Kyaw were Richa Gupta (architecture) and Bradley Bunch, Nidhish Sagar, and Michael Won — all from the MIT Department of Electrical Engineering and Computer Science (EECS).
Curator AI is designed to streamline online furniture shopping by providing context-aware product recommendations using AI and AR. The platform uses AR to take the size of a room with locations of windows, doors, and existing furniture. Users can then speak to the software to explain what latest furnishings they need, and the system will use a vision-language AI model to go looking for and display various options that match each the user’s prompts and the room’s visual characteristics.
“Shoppers can pick from the suggested options, visualize products in AR, and use natural language to ask for modifications to the search, making the furniture selection process more intuitive, efficient, and personalized,” Kyaw says. “The issue we’re trying to resolve is that almost all people don’t know where to start out when furnishing a room, so we developed Curator AI to supply smart, contextual recommendations based on what your room looks like.” Although Curator AI was developed for furniture shopping, it might be expanded to be used in other markets.
One other example of Kyaw’s work is Estimate, a product that he and three other graduate students created throughout the MIT Sloan Product Tech Conference’s hackathon in March 2024. The main target of that competition was to assist small businesses; Kyaw and team decided to base their work on a painting company in Cambridge that employs 10 people. Estimate uses AR and an object-recognition AI technology to take the precise measurements of a room and generate an in depth cost estimate for a renovation and/or paint job. It also leverages generative AI to display images of the room or rooms as they could appear like after painting or renovating, and generates an invoice once the project is complete.
The team won that hackathon and $5,000 in money. Kyaw’s teammates were Guillaume Allegre, May Khine, and Anna Mathy, all of whom graduated from MIT in 2024 with master’s degrees in business analytics.
In April, Kyaw will give a TedX talk at his alma mater, Cornell University, wherein he’ll describe Curator AI, Estimate, and other projects that use AI, AR, and robotics to design and construct things.
Considered one of these projects is Unlog, for which Kyaw connected AR with gesture recognition to construct a software that takes input from the touch of a fingertip on the surface of a cloth, and even within the air, to map the size of constructing components. That’s how Unlog — a towering art sculpture created from ash logs that stands on the Cornell campus — got here about.
Gesture Recognition for Feedback-Based Mixed Reality and Robotic Fabrication of the Unlog Tower
Video: Alexander Htet Kyaw
Unlog represents the chance that structures could be built directly from an entire log, reasonably than having the log travel to a lumber mill to be was planks or two-by-fours, then shipped to a wholesaler or retailer. It’s a great representation of Kyaw’s desire to make use of constructing materials in a more sustainable way. A paper on this work, “Gestural Recognition for Feedback-Based Mixed Reality Fabrication a Case Study of the UnLog Tower,” was published by Kyaw, Leslie Lok, Lawson Spencer, and Sasa Zivkovic within the Proceedings of the fifth International Conference on Computational Design and Robotic Fabrication, January 2024.
One other system Kyaw developed integrates physics simulation, gesture recognition, and AR to design lively bending structures built with bamboo poles. Gesture recognition allows users to govern digital bamboo modules in AR, and the physics simulation is integrated to visualise how the bamboo bends and where to connect the bamboo poles in ways in which create a stable structure. This work appeared within the Proceedings of the forty first Education and Research in Computer Aided Architectural Design in Europe, August 2023, as “Energetic Bending in Physics-Based Mixed Reality: The Design and Fabrication of a Reconfigurable Modular Bamboo System.”
Kyaw pitched the same idea using bamboo modules to create deployable structures last 12 months to MITdesignX, an MIT MAD program that selects promising startups and provides coaching and funding to launch them. Kyaw has since founded BendShelters to construct the prefabricated, modular bamboo shelters and community spaces for refugees and displaced individuals in Myanmar, his home country.
“Where I grew up, in Myanmar, I’ve seen numerous day-to-day effects of climate change and extreme poverty,” Kyaw says. “There’s an enormous refugee crisis within the country, and I would like to take into consideration how I can contribute back to my community.”
His work with BendShelters has been recognized by MIT Sandbox, PKG Social Innovation Challenge, and the Amazon Robotics’ Prize for Social Good.
At MIT, Kyaw is collaborating with Professor Neil Gershenfeld, director of the Center for Bits and Atoms, and PhD student Miana Smith to make use of speech recognition, 3D generative AI, and robotic arms to create a workflow that may construct objects in an accessible, on-demand, and sustainable way. Kyaw holds bachelor’s degrees in architecture and computer science from Cornell. Last 12 months, he was awarded an SJA Fellowship from the Steve Jobs Archive, which provides funding for projects on the intersection of technology and the humanities.
“I enjoy exploring different sorts of technologies to design and make things,” Kyaw says. “Being a part of MAD has made me take into consideration how all my work connects, and helped make clear my intentions. My research vision is to design and develop systems and products that enable natural interactions between humans, machines, and the world around us.”
