7 ms·

I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a se

I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a semantic data modeling API will yield a more useful/practical result. You may even find the semantic layer makes a natural language approach moot in many cases. In Zillion* I added an experimental feature that uses OpenAI to form a report API call from natural language. Fine tuning (vs prompt tuning) this on the semantic model with gpt-3.5 would probably yield some notable improvement in abilities, as I view it as more of a toy feature at the moment. * https://github.com/totalhack/zillion https://github.com/totalhack/zillion

antupis 3y ago
Yeah it is literally billion dollar question how to do that querying. Is there some good papers or blog posts that go through different approaches?
- totalhack 3y ago
  I think there are probably a million posts and papers talking about text to SQL as it's a problem that's been tackled for a long time now...just with more vigor recently now that LLMs provide an improved/shiny approach. I think many companies now offer or consider themselves semantic layers in some fashion (google "semantic layer" + tableau, cube.dev, dbt...). I have not come across any blog or paper comparing text-to-sql directly to text-semantic-api. Certainly the latter is not as flexible as the former in that it can't produce every possible SQL query. In practice (for me at least) that's often not necessary, or is necessary rarely so you just do your direct SQL as needed and use your semantic layer >95% of the time.
noorvir 3y ago
Can you describe in more detail what you mean by semantic? Doesn’t the DB schema capture the business logic to a large extent?
- totalhack 3y ago
  From our good friend GPT-4 who can summarize it better than I can... A semantic data layer is an abstraction layer that sits between the raw data stored in databases and the applications or analytics tools that use this data. The purpose of this layer is to provide a consistent, business-friendly interface to data that might be stored in a variety of formats, tables, or systems. By doing so, the semantic data layer helps in simplifying complex data structures into more meaningful, understandable models. Key Features: Unified View: It provides a unified view of data from multiple sources, making it easier for users to access and understand the data without having to know the underlying structure or complexity. Data Governance: It can enforce business rules, security policies, and data quality measures, ensuring that data is consistent, compliant, and accessible only by authorized users. Flexibility: A semantic layer is often designed to be flexible, allowing business users to adapt to changes in the data model without requiring changes to the applications themselves. Query Simplification: The layer simplifies the process of querying data by providing a more user-friendly way of accessing and manipulating data, often through a drag-and-drop interface or other graphical tools. Decoupling: It decouples application development from data source changes. When underlying data changes, you don't necessarily have to update all the applications that use it; you might only need to update the semantic layer. Data Integration: It can integrate data from multiple sources, providing a single "source of truth" for business users and applications. Purpose: Simplification: Make data more accessible and easier to understand for non-technical users. Consistency: Ensure that everyone is working from the same definitions and business rules. Efficiency: Reduce the time and complexity involved in generating reports, analytics, and other data-driven functions. Data Quality: Help to enforce data quality and governance policies. Security: Provide a mechanism for enforcing security rules on who can see or modify data. Adaptability: Enable quicker adaptation to changes in business needs or data structures. By providing a semantic data layer, organizations can ensure that their data is not only high-quality and secure but also that it can be easily used for making informed decisions. This is particularly valuable in today's data-driven business environment.

antupis 3y ago

Yeah it is literally billion dollar question how to do that querying. Is there some good papers or blog posts that go through different approaches?

totalhack 3y ago
I think there are probably a million posts and papers talking about text to SQL as it's a problem that's been tackled for a long time now...just with more vigor recently now that LLMs provide an improved/shiny approach. I think many companies now offer or consider themselves semantic layers in some fashion (google "semantic layer" + tableau, cube.dev, dbt...). I have not come across any blog or paper comparing text-to-sql directly to text-semantic-api. Certainly the latter is not as flexible as the former in that it can't produce every possible SQL query. In practice (for me at least) that's often not necessary, or is necessary rarely so you just do your direct SQL as needed and use your semantic layer >95% of the time.

noorvir 3y ago

Can you describe in more detail what you mean by semantic? Doesn’t the DB schema capture the business logic to a large extent?

totalhack 3y ago
From our good friend GPT-4 who can summarize it better than I can... A semantic data layer is an abstraction layer that sits between the raw data stored in databases and the applications or analytics tools that use this data. The purpose of this layer is to provide a consistent, business-friendly interface to data that might be stored in a variety of formats, tables, or systems. By doing so, the semantic data layer helps in simplifying complex data structures into more meaningful, understandable models. Key Features: Unified View: It provides a unified view of data from multiple sources, making it easier for users to access and understand the data without having to know the underlying structure or complexity. Data Governance: It can enforce business rules, security policies, and data quality measures, ensuring that data is consistent, compliant, and accessible only by authorized users. Flexibility: A semantic layer is often designed to be flexible, allowing business users to adapt to changes in the data model without requiring changes to the applications themselves. Query Simplification: The layer simplifies the process of querying data by providing a more user-friendly way of accessing and manipulating data, often through a drag-and-drop interface or other graphical tools. Decoupling: It decouples application development from data source changes. When underlying data changes, you don't necessarily have to update all the applications that use it; you might only need to update the semantic layer. Data Integration: It can integrate data from multiple sources, providing a single "source of truth" for business users and applications. Purpose: Simplification: Make data more accessible and easier to understand for non-technical users. Consistency: Ensure that everyone is working from the same definitions and business rules. Efficiency: Reduce the time and complexity involved in generating reports, analytics, and other data-driven functions. Data Quality: Help to enforce data quality and governance policies. Security: Provide a mechanism for enforcing security rules on who can see or modify data. Adaptability: Enable quicker adaptation to changes in business needs or data structures. By providing a semantic data layer, organizations can ensure that their data is not only high-quality and secure but also that it can be easily used for making informed decisions. This is particularly valuable in today's data-driven business environment.