Data ingestion

Data ingestion

The process of collecting, cleaning, and feeding raw data into a system so it can be used for training an AI model or retrieved by one, such as scraping text from the web or importing a set of documents into a retrieval-augmented generation system. Poor-quality or poorly filtered ingestion is a common source of downstream problems, from bias to factual errors. (See also: Training data, Retrieval-augmented generation)