6 ms·
Text2CAD: Generating sequential cad designs from text prompts
- timonoko 2y agoGemini clearly understands OpenScad primitives. "Make this object hollow", etc. But "Make me a teapot" seems to be aiming for teapot-looking object, which cannot hold tea. But this is not failure perse.
- timonoko 2y agoMaking functional teapot takes about 10 queries. You just have to make the pot and spout hollow and move the handle outside. Totally functional tea-pot production solution already. All you need to do is to automatize the display function and STL-production and any granny can print her own teapot without understanding nothing about computers and 3D-printers.
- waciki 2y agoThose grannies are already making teapots, it's called pottery.
- throwgfgfd25 2y agoMore than a thousand generations of grannies, at that. Additive manufacturing from 30,000 years ago!
- deleted 2y ago[deleted]
- jsheard 2y agoRest in peace granny, taken from us by a cocktail of plastics, bacteria, mold and tea. Maybe stating the obvious but 3D printed parts usually aren't food safe unless you post-process them with an appropriate non-porous coating.
- andai 2y agoWhat's your favorite non-porous teapot coating?
- timonoko 2y agoNo coating needed. All you need is to re-flow the ABS. Feels like pottery after this simple operation: https://youtu.be/iLaGJwCCz-E?si=a1gccp8EI9sGzt-e https://youtu.be/iLaGJwCCz-E?si=a1gccp8EI9sGzt-e
- echoangle 2y agoDo you really want almost-boiling-water in ABS? I wouldn't do that.
- timonoko 2y agoReflow happens at 200°C, out gassing any 3D-related harmful stuff. Equal to inject-molded ABS cutlery.
- echoangle 2y agoMaybe I'm too careful but I wouldn't use plastic cutlery on hot food either. Also, the glass transition temperature of ABS is only 105 °C, so It's probably not too strong when filled with 100 °C water.
- timonoko 2y agoI have already made all kinds of tests. Only restriction was that you cannot make open flame cooking pot, because it starts combusting too easily.
- echoangle 2y agoHow did you test the water quality after storing it in an ABS container? How would you notice if something from the ABS dissolved in the water?
- jaakl 2y agoDoes it respond to http queries?
- avodonosov 2y agoWhat format for resulting CAD designs they use? And in general, what is (are) the best open CAD formats and applicatioms today?
- iancmceachern 2y agoThis is the rub
- deleted 2y ago[deleted]
- q3k 2y agoDescribing what I want to design seems like more effort than actually just sketching things out in a decent CAD package.
- serf 2y agoonly if you're familiar with the CAD package. presumably a big benefit here is that all it really takes English and geometry knowledge.
- Palomides 2y agothinking in terms of composing 3d objects and their positions is 90% of doing CAD already, if you can do that, you can reproduce any of the objects in OP with 15 minutes of learning the tool seriously, I think people overestimate how hard basic CAD work is
- iancmceachern 2y agoYeah, if you architect your part right, which takes experience. For folks that don't belive us, check out "speed modeling"
- throwgfgfd25 2y agoIt does take experience but mostly it takes a little analysis. I'm still really quite green at CAD; I've come to it late in life and I only make relatively simple things, perhaps; only simple mechanisms. But when I look at people getting stuck and asking for help in CAD groups it so often comes down to the knock-on effects of very early mistakes, like squandering the benefits of the base planes by choosing the wrong initial orientation, muddling through with primitives when an extrusion of a sketch would do the job, or making a series of complex circular pockets when a single revolve could have done it better. Basic familiarity with a few principles and their expression in CAD, and a little study of existing objects gets you a long way. It interests me that programmers are willing to learn the expressive nuances of individual languages or libraries or methodologies, but as soon as it comes to GUI CAD they dismiss the whole thing as too hard or too obscure. The excitement around LLM text-to-CAD seems emblematic of this; as if the wrong conclusions about GUI interfaces have been drawn from bad experiences of dev GUIs.
- kkfx 2y agoThe idea is very nice, but as all LLM-backed stuff I see results are poor, meaning we need more effort to use the LLM obtaining good results than doing the work with non-LLM-wrapped/backed tools. It's the same for copilot to code and so on. It's still an interesting applications that might have a better future concentrating toward simulations (Salomé MECA, CodeASTER etc) since for these systems it's more tedious designing the simulation by hand than just sketch a 3D part with a modern parametric CAD.
- l5870uoo9y 2y agoThat's something artificial intelligence is really good at: fitting data into a formalized format. I'm working on something similar[0], here the goal is not CAD design but charts. This is also something that has only become possible with later versions of AI, such as GPT-4. [0]: https://www.datavisualizer.ai/ https://www.datavisualizer.ai/
- faeyanpiraat 2y agoDo you have paying users?
- l5870uoo9y 2y agoNo, it is still in beta and I haven't actually launched it. Did you like it?
- echoangle 2y agoAnyone who has ever done CAD knows that a picture is worth a thousand words. Describing a 3D object you want in words is much more effort to get right than drawing a simple sketch. Wouldn’t it be better to have image input for an application like this?
- DonnyV 2y agoYeah, a way to turn a flat sketch to a 3D model would be a better way to do this.
- jsheard 2y agoYeah but when all you have is an LLM hammer, everything looks like a text2text nail.
- CSMastermind 2y agoLLMs are fundamentally one dimensional which works fine when you're generating next tokens for text which because that's a 1D problem. I do wonder how much progress we could make on a problem like this with a 3D transformer architecture.
- abotsis 2y agoI’m not sure I follow this. Isn’t an LLMs dimensionality measured by how many parameters the model supports? Ie 10s of billions in some cases? If I understand it correctly, then, the model is already evaluating things in lots of dimensions and reducing it down to 1, as you say in the case of text, 2 dimensions in image generation, 3 should be pretty straightforward.
- btbuildem 2y agoI think they're referring to the dimensionality of the input / output space, not the intermediate internal representation.
- btbuildem 2y ago
- isoprophlex 2y agoI wonder how effective a solution leveraging LLMs would be in producing NC code... maybe a numerically precise method is still a hard requirement there.
- iancmceachern 2y agoWe don't need LLM for that, and it would be very risky to trust not crashing a $250k machine tool with a $5k part in it that could kill the operator if something goes wrong
- therouwboat 2y agoThere is AI assistant for mastercam and other cam software, it promises to make 80% finished programs, but parts look really simple, like something you do on the first week of cnc training.
- sgnelson 2y agoMastercam already has Automatic Tool Pathing. It's essentially click "go." It'll spit out all the toolpaths to make the part. It'll be inefficient and if you don't tell it specifically what tools to use, it'll use 100 different tools, but it's actually not too bad if you set it up correctly. A number of other CAM packages have similar features now.
- loughnane 2y agoThis would be most useful if it could generate objects that could then easily be tweaked, ideally parametrically. I think with a long preamble in the prompt (you are an expert at designing injection molded parts, and so on) you could get something pretty useful.
- westurner 2y agoFrom https://news.ycombinator.com/item?id=40131766 https://news.ycombinator.com/item?id=40131766 re: LLM minifigs and parametric CAD parts libraries: > Is there a blenderGPT-like tool trained on build123d Python models? > ai-game-development tools lists a few CAD LLM apps like blenderGPT and blender-GPT: https://github.com/Yuan-ManX/ai-game-development-tools#3d-model https://github.com/Yuan-ManX/ai-game-development-tools#3d-mo...
- magic_man 2y agocan you import this into solid works or autocad? I think it would be super useful to have the basic model and then tweak it.
- throwgfgfd25 2y agoThis is one of those classic LLM scenarios that doesn't make sense. If you can tweak the model in a CAD package, you can quite possibly make it in CAD in the first place more quickly than you can manage several rounds of descriptions with an LLM. It's like the whole "I'd like to get an LLM to write a song and then I'd adjust it" thing. No musician needs this. If you can truly fine-tune a song, writing one is easy. And if you aren't a musician, you won't get good results fine-tuning a song.
- qazxcvbnmlp 2y agoSo much negativity here. It’s interesting to see new technical possibilities. Congratulations to the team that made this.
- throwgfgfd25 2y agoIt's not negativity so much as informed dissent.
- Aurornis 2y agoCool in theory, but I can’t imagine describing anything other than the most basic CAD designs verbally. Text is not a good medium for designs beyond the most basic shapes and modifications.
- CSMastermind 2y agoI work with software that displays 3D models of real products. If I could feed the manufacturers spec sheet along with maybe some pictures of the items and marketing copy then get a 3D model out it would save me a ton of money.
- benmccann 2y agoWhat software is that?
- jareklupinski 2y ago98% of the total time it takes me to design something, usually involves deleting entire designs because they get a point where something is unfeasible, invalid geometry, or against all manufacturers' guidelines (will this shape work with a draft angle? etc.) this is usually before i start adding intricacies such as shells, fillets, and other features, which do take a lot of effort, but making those by hand is more of the 'art' side of the process anyway anything that gets me through the first 98% is welcome :)
- throwgfgfd25 2y ago> anything that gets me through the first 98% is welcome :) Only if the result of the LLM is a good, well-architected parametric CAD model you can adjust, right?
- jareklupinski 2y agothat seems like a little much to expect from an LLM; the average CAD file in my experience has not been not well-architected :) as long as the output is something my manufacturer can understand (downloadable mesh: STL/STEP/etc (they dont take links)), the tool did its job for me i would probably start the final model from scratch no matter what the output was, so i can match my chosen manufacturer's tolerances/design rules/optimizations, and to give breathing room for my quirks/workflow (i like to design subtractively, some people design additively)
- JofArnold 2y ago> usually involves deleting entire designs Yes, for me that's usually around the point I've got a huge and complex assembly with all the motion wired up. Right-click -> Duplicate -> Rename "MyProject V2". I would save a huge amount of time also this way.
- btbuildem 2y agoI see a lot of comments here to the effect "but it's easier to just do it in CAD in the first place" -- those are experts speaking. Imagine being a newcomer to CAD, the learning curve is quite steep. Compared to mastering the complex UI and workflows of a CAD problem, a text-based approach seems much easier. I can see this being incorporated into existing software as an alternate workflow path. What caught my curiosity here is the "sequential" qualifier. One massive weakness of all the AI content generation schemes is the lack of editability -- you get what you get, and attempts to refine the initial results are middling at best. This seems like it allows the user to build a more complex scene from multiple prompts -- likely meaning you can go back and edit some of the prompts to tweak the building blocks, and edit the overall scene. Interesting!
- throwgfgfd25 2y ago> I see a lot of comments here to the effect "but it's easier to just do it in CAD in the first place" -- those are experts speaking. This is what I think, and yet I am the longest time away from being an expert. It's easier because CAD tools unlock CAD thinking and empower your brain to actually do design, as well as helping you see problems you didn't anticipate. This whole area -- LLM to CAD -- is one of the most misguided applications for generative AI (beyond "generative design" as it was understood in the pre-LLM/pre-GAN era, where it was usually used for FEM/topology optimisation) There are already enormous libraries of freely available basic CAD models for real-world objects; any beginner would be much better off simply learning how to merge them. And any tool aimed at beginners would be better off assisting that process (TinkerCad does, for example) And if a beginner has a truly novel object to make, an LLM is not going to have the training set data to make it. Nor is the beginner likely to have the CAD knowledge (words, expressions) to describe it. For this to be of use to a beginner you do have to imagine quite a niche kind of beginner: one who is expert in descriptive language and advanced geometry. Those people would be better off learning some sort of CAD environment; indeed they are the niche that is least likely to be driven insane by the limitations of OpenSCAD.
- userbinator 2y agoI wouldn't consider CAD UI to be "complex". It's not too different from any vector image editor, except for a third dimension.
- maCDzP 2y agoLet’s see where this goes in 1-2 years. There is a ton of money in the CAD industry.
- mwill 2y agoI see multiple comments arguing that using a CAD package is only easier and faster if you already know how to use a CAD package, and this is a 'better' UI for people who don't have those skills....but in that scenario, are you not then just trading the fixed upfront time investment to learn the basics of CAD, for ongoing inefficiency and difficulty every time you want to model something? For a user of a UI like this, there comes a point where their time would have been better spent learning a CAD package. Another layer is if you are modelling something that has to be machined or built in real life, you have to be keeping an eye on how it will physically exist throughout the entire process, stock it will be machined from or materials it will be built with. Thinking in terms of CAD workflows help with this greatly in my experience. The operations shown in the demo are not only easier to perform in a CAD package than describe in English to an LLM, but also the easiest part of it (except maybe if you are designing strictly for 3D printing)
- dingnuts 2y ago> For a user of a UI like this, there comes a point where their time would have been better spent learning a CAD package. for some users, they will think of few enough designs in a lifetime to make learning any specific software worthwhile. For these users, the LLM's inefficiencies are worth the trade-off.
- throwgfgfd25 2y agoBut the thing is, those users are unlikely to have thought of anything novel simply because they are not designers: if the tool is going to be successful then what they want is likely to be in the training set and easily googleable. This whole idea seems contingent on imagining a situation where a non-CAD user has an idea for a truly novel physical object, has extensive geometry skills, and can describe that object in some magical level of detail that doesn't involve any terms of art from the CAD domain. It's not a very likely scenario. And the energy put into tools to support this scenario would be better spent improving searchability of the data that is going to go into the training set, and simple tools to allow objects in the training set to be combined (such as those offered by TinkerCad or Microsoft's 3D builder). It's also prone to the risk that the LLM gets something wrong: makes a part that is prone to failure, or will actually destroy a CNC, or be unprintable by a 3D printer, etc.
- sanchezxg 2y agoAlright
- sgnelson 2y agoBecause a lot of people are saying "CAD is only useful if you know how to use it." One word: Tinkercad. https://www.tinkercad.com/ https://www.tinkercad.com/ I teach children how to use it. It takes them about 15 minutes to pick it up. Are you going to design a car with it? No. But if you look at all the Text2Cad programs so far, they won't either. If you need to design something for 3d printing, Tinkercad, in terms of ease of use/simplicity, is hard to beat. And it's free to use. I've played with other Text2Cad projects, and I've yet to be able to get anything out of them other than the most simple shapes, which frankly, what's the point if it can't make an object that's useful or meets my requirements? It takes way too much time having to write out paragraphs to even get the most basic of brackets built. I remember on the Zoo app, it very much likes typical CAD language which means you're going to need to be able to speak "CAD" anyways when trying to create a successful part (think: constraints in typical CAD) What these LLM programs really need to focus on (imo) is writing Openscad (and similar "programmable CAD") code, which is going to at least give the user a better starting place, as well as the ability to edit their parts. I think one of the biggest constraints for this is the lack of millions of lines of code and documentation unlike most popular programming languages for the LLM. I think the research is neat, but for now, it seems like a solution in search of a problem.
- xunil2ycom 2y agoI started years ago with Tinkercad. Eventually I needed more and moved to Fusion 360. It's a natural progression. Tinkercad was a great place to start. Better than Text2CAD (sorry to the creators).
- deleted 2y ago[deleted]
- mdsadilkhan99 2y agoHello everyone, I'm the lead author of Text2CAD, and I’d like to start by expressing my gratitude for all the support and constructive criticism on our paper. As an AI researcher, it's essential to engage with diverse perspectives, as these discussions spark new ideas for improvement and growth. I would like to answer a few questions raised in the discussion. Final CAD Representation: Our CAD model representation is inspired by Onshape's Featurescript, a code-based CAD model generator. We opted for this since the available open-source datasets are from Onshape. The final CAD model is output in a sequence which can be transformed to .stl or .step format, using PythonOCC. These sequences are editable as it contains parameters for sketch and extrusion. In the future, as Text2CAD matures, we plan to support direct Onshape Featurescript generation as well. We're also planning to release a demo on HuggingFace soon, free for anyone to use. I would love to collaborate with the community for Onshape Featurescript generation, so if anyone is interested in contributing, feel free to reach out via email! LLM Usage: I want to clear up any confusion around how we use large language models (LLMs). Right now, LLMs aren’t involved in generating the final CAD model. Instead, we use them to help create the dataset (for example, text annotations). The actual CAD generation is powered by a Transformer-based architecture that works in a sequence-based approach (sketch-and-extrude). That said, integrating LLMs into the CAD generation process could very well become a reality in the future, and we’re keeping a close eye on that evolution. Simpler Shapes: Some people have pointed out that Text2CAD currently generates simpler shapes, and I agree. This is mainly due to the limitations of the datasets (not the method itself) we’re working with, which are restricted in complexity. We can currently only handle sketches with basic elements like lines, arcs, and circles, alongside extrusion operations. The creation of new CAD datasets is challenging due to IP restrictions, and as many of you know, great AI systems are built on strong datasets. So it will take time for open-source research in CAD to be fully usable for application. However, we’re hopeful that as more complex datasets become available, you’ll see far more advanced possibilities from Text2CAD and its successors. Theory vs Practical (What is Text2CAD's contribution ??): As an AI researcher, it’s crucial for us to balance theoretical contributions with practical applications. Translating geometric and parametric information from text into 3D models is a significant open problem in NLP. With Text2CAD, we’ve taken a step toward addressing this by providing a dataset that others can leverage for further research. I understand the sentiment that "a picture is worth a thousand words"—the future of CAD is undoubtedly multi-modal, much like GPT-4, where both language and images will coexist. For this to happen, AI agents must deeply understand both modalities. Ultimately, it comes down to personal preference: whether a user finds working with text or images more intuitive. This question is just as relevant for other text-to-3D projects as well. Assistance vs Replacement: In my website, I use the term “AI-Assisted CAD Modeling” to emphasize that the goal of our research is to support CAD designers, not replace them. The primary aim of Text2CAD is to streamline and speed up the prototyping process, rather than creating complete models from scratch. I recognize that not everyone may enjoy specifying numerous parameters, so we've also included the option to work with shape concepts without parameters. As with many tools, it comes down to personal preference, whether one prefers a more hands-on approach or quicker, conceptual modeling. Future Goal: Looking ahead, I’d love to hear from you —especially from CAD designers— about how you'd like to see AI assist you in your CAD designing process. Your opinions are important for AI researchers like myself, as they help us design tools that are truly useful and aligned with real-world needs. Reach out to me if you are interested. Cheers.