Fusion 360 Gallery Dataset Pushes the Boundary of 3D Geometry Generation
In a paper published on arXiv on October 5, researchers from Autodesk AI Lab and MIT described the motivations behind releasing the dataset, along with standard CAD reconstructions tasks, evaluation metrics, and novel methods to recover construction sequence from raw geometry. The Fusion 360 Gallery reconstruction dataset contains construction sequence information for 8,625 CAD models created by users. It is the first dataset to provide human-designed 3D CAD construction sequence for use with machine learning. Additionally, reconstruction tasks and evaluation metrics provide other researchers with baselines to test new algorithms and provide a universal benchmark for learning-based CAD reconstruction algorithms.
The DatasetThe reconstruction dataset was created using designs submitted by users of Autodesk Fusion 360, an integrated CAD, CAM, CAE, and PCB software that allows users to easily collaborate in an integrated environment. The dataset contains CAD construction sequence information from a subset of ‘sketch and extrude’ designs. The data is provided in three different formats: b-rep, mesh, and construction sequence JSON text.
Boundary representation (B-reps) is the preferred format for parametric CAD. B-reps represent solids as a collection of connected surface elements and their boundaries. These contain topological information that describes adjacency relationships between surfaces, loops, edges, and vertices. Although b-reps provide a high level of control of 3D shapes, they lack construction sequence information. Knowing how a design came about is crucial for learning-based algorithms, making the Fusion 360 Gallery dataset a valuable resource.
Fusion 360 GymIn the paper, the researchers also introduce an environment called Fusion 360 Gym, which standardizes the CAD reconstruction task in a Markov Decision Process (MDP). An MDP is a model that describes sequential decision-making processes. MDPs have been widely used in the field of Reinforcement Learning (RL), as they are a straightforward framing of the problem of learning from interactions with an environment to achieve a goal. In the case of the Fusion 360 Gym, an intelligent agent has the goal of CAD reconstruction. Therefore, the MDP provides the current state of the geometry and target geometry to be reconstructed, actions that the agent can take, such as sketch extrusion and face extrusion, transitions that apply the actions to the current and a reward that compares the current and target geometry.
The researchers have also defined a set of tasks along with evaluation metrics that can be used as baselines for other researchers to test new algorithms for CAD reconstruction. They also provide a method for CAD reconstruction based on a neurally-guided search. The dataset, tasks, evaluation metrics, and reconstruction method are an important step in advancing the field of 3D shape generation and will help accelerate the development of new algorithms and provide powerful new tools for designers and engineers.