Data preparation to make internal company documents usable by AI refers to the work of taking inventory of scattered regulations, meeting minutes, and manuals, and organizing their confidentiality classifications, formats, and metadata so they can be fed into RAG. When information systems departments or DX promotion personnel introduce an in-house AI assistant, they often struggle to decide where to start. This article explains, from a practical perspective, everything from how to select priority areas to how to systematize update operations. Trying to organize all documents at once tends to bring the work to a halt, but by starting with the areas that are used most frequently, it becomes easier to achieve both search accuracy and speed of implementation.

RAG is a mechanism that combines search and generation. Therefore, if the documents being fed in are outdated, duplicated, or contain a mix of confidential information, the search stage will pick up noise, and the generated answers will likely miss the mark. For example, if an old regulation whose updates stopped six months ago remains in the same folder as the latest version, the AI cannot determine which one to use as the basis, and may confidently present the outdated information. Insufficient data preparation is not merely a cosmetic issue—it directly affects the reliability of the answers themselves. Let us first look, in order, at the specific harms that insufficient preparation can cause.

RAG is a mechanism that vectorizes the question text and document fragments, searches based on semantic proximity, and generates an answer. The foundation of this search is the documents that have already been ingested, and no matter how much the model's performance is improved, if the original documents are outdated, duplicated, or have inconsistent notation, the system cannot extract the appropriate fragments. The relationship "garbage in, garbage out," as applied to search engines, holds true here.

If the same regulation remains in two versions—pre-revision and post-revision—the search will judge both as "closely relevant," making it impossible to distinguish which one should take priority. When formats such as Word, PDF, and scanned images are mixed together, the accuracy of text extraction varies by document, resulting in some documents with identical content being easier or harder to retrieve via search. In cases where the file name or headings alone do not clarify a document's positioning, search tends to rely on literal text matches within the body, which reduces the system's ability to handle rephrased questions.

All of these are problems that can be resolved through preparation at the stage before documents are ingested. In fact, removing these inconsistencies during the early stages of preparation has a greater impact on search accuracy than later-stage improvements such as chunk splitting or search algorithm refinement. Whether these three tasks—consolidating revision history, unifying formats, and adding meta-information indicating positioning—are completed beforehand makes a significant difference in the quality of search results, even within the same RAG framework. Establishing a solid foundation on the document side is the first step toward effective AI utilization.

When RAG is put into operation without sufficient preparation, the same kinds of symptoms consistently appear.

The most common is a case where outdated regulations or manuals are returned as search results. If both the pre-revision and post-revision documents remain in the knowledge base, the AI cannot determine which one is valid, and may answer based on an old leave policy or outdated rate schedule. This symptom is especially likely to occur in documents lacking metadata such as revision dates or expiration dates. In particular, HR and general affairs regulations undergo minor revisions several times a year, so it is common for old versions to remain undeleted.

The next most common issue is answers that mix content from multiple documents. When files with similar names, or procedure manuals that differ slightly by department, coexist, search results may be combined in a fragmented way, generating procedures that do not actually exist. A typical example is when the application flow for the Sales Department and the application flow for the Administration Department exist as separate files, both saved under the title "Application Procedure"—the AI may stitch together parts of both to create a fictitious procedure.

The most serious issue is an incident in which HR evaluations or contract terms that unauthorized employees should not see end up mixed into an answer. This occurs because all documents were ingested without organizing confidentiality classifications, and it is not merely a matter of answer accuracy—it directly leads to real damage in the form of information leakage. If HR evaluation sheets marked "for executives and officers only," or lists of contract terms that differ by business partner, are placed in the knowledge base without classification, there is a risk that such content will be directly quoted in response to a question from a general employee.

What these three symptoms have in common is that the model's performance

Trying to organize all documents at once causes the workload to balloon, often bringing the implementation itself to a halt. It is not uncommon for organizations to attempt to survey hundreds of folders all at once, only to have months pass with nothing actually done. What should be prioritized is identifying the areas with high usage frequency and pinpointing confidential documents that must not be fed to the AI. Here, we organize the perspectives needed to narrow down the scope.

If all documents are treated as targets for inventory, the work will never finish and the implementation plan will stall. In practice, it is more effective to narrow down priority areas using two axes: "frequency of use" and "frequency of updates."

Documents with high frequency of use refer to areas where inquiries and searches are concentrated. Employment regulations, expense reimbursement rules, product specifications, and FAQ collections often fall into this category. Organizing these areas makes it easier for users to feel the benefits early on. Conversely, old proposals or past meeting materials that are referenced only a few times a year have little impact on operations even if left for later.

Update frequency affects priority from a different angle. Regulations and manuals that are revised frequently become outdated shortly after being organized, so introducing them before an update mechanism is in place can become a breeding ground for misinformation. On the other hand, documents that are stable and rarely updated, such as laws and internal regulations, offer high return on investment since organizing them once allows for long-term use.

Combining these two axes reveals the order in which to proceed. Documents with "high frequency of use and low frequency of updates" are the top priority area, and employment regulations and internal company rules often fall into this category. For documents with "high frequency of use but also high frequency of updates," it is better to first establish an update management system rather than rushing to organize them. Skipping this step and introducing them anyway can result in the content diverging from reality within a few months, causing the AI to answer with outdated information. Conversely, documents that are low on both axes can be temporarily excluded from the organization process with little impact on operations. A realistic approach is to start with the "high frequency of use, low frequency of updates" area first, confirm results, and then gradually expand the scope.

One aspect often overlooked in document organization is the selection of documents that should not be shown to the AI. If all documents are input into RAG, there is a risk that information meant to be accessible only to a limited number of people may end up mixed into search results. This is akin to laying out the entire contents of a locked room in the hallway—an area where convenience and safety can easily trade off against each other.

When designing confidentiality classifications, first determine levels such as "publicly available," "internal use only," "specific department only," and "access prohibited," and assign labels to each document accordingly. Documents containing personnel evaluations, salary information, contract terms with business partners, or personal information should, in principle, be placed in the category not shown to the AI. When handling information related to individuals, it is also useful to determine whether names, contact information, and similar identifiers can be mechanically removed, referring to how identifiers are treated under guidelines for anonymized and pseudonymized information.

Documents with high reference frequency across the entire company, such as regulations and manuals, are relatively easy to classify as public. In contrast, contracts and personnel-related documents tend to have confidentiality levels that vary depending on individual circumstances, and perceptions of how they should be handled often differ by department. In the early stages of classification design, a conservative approach is realistic: temporarily exclude documents whose classification is unclear, and begin with those whose public classification is clear. Excluded documents should not simply be left unattended—without deciding who will determine their classification and by when, they will remain indefinitely in a "pending" state.

Classification rules are not something decided once and then finished. Designing the system with the assumption that classifications will be reviewed whenever there are department transfers or organizational changes—checking whether documents whose responsible party has changed still carry outdated classifications—helps keep the operational burden manageable going forward.

Dividing the process from inventory to search testing into five stages makes it easier to prevent omissions in the work. First, conduct inventory and remove duplicates, then proceed with format standardization and metadata assignment, and finally finish with search testing. Following this order helps prevent the scope of the target from shifting midway. In particular, if the scope is not properly narrowed down during the initial inventory stage, rework tends to occur across all subsequent processes, making it worthwhile to spend adequate time here. The following H3 sections will look at the specific approach for each step in order.

The first step of inventory is to identify all documents that exist within the target area and compile them into a list. It is common for storage locations to be scattered across file servers, shared drives, internal company wikis, and email attachments, so the scope of organization cannot be defined without first visualizing "what exists where."

The identified documents should be compiled into an inventory list arranged by title, storage location, last updated date, and creating department. At this stage, it often becomes apparent that the same regulation has been saved under different names in multiple locations, or that both the old and revised versions still remain. When you actually go through this process, you will almost always encounter cases where "a file believed to be the latest version turned out to actually be three generations old." It is not uncommon for the memory of those in charge to diverge from reality.

If duplicates and outdated versions are input into RAG as they are, the AI will reference both old and new information, resulting in contradictory answers. This is the most labor-intensive part of the entire organization process, and if handled incorrectly, all subsequent steps—format standardization and metadata assignment—will require rework. For documents with identical content, keep the one with the most recent update date and delete or archive the others. For documents lacking an effective date or revision history, check with the responsible department to confirm whether it is the current version. Documents that exist only on an individual's personal computer require a decision at this stage as to whether they should be treated as an official management target. In particular, for documents originating from personal computers, often only the creator knows the background, so postponing confirmation can make it impossible for anyone to make that determination later.

Completing this inventory and removal process beforehand allows the scope of the subsequent format standardization and metadata assignment work to be narrowed down. Conversely, proceeding to the next process without carefully completing this step will result in having to redo the entire organization process.

Once the inventory is complete, the first step is to standardize the format of the documents. If Word, PDF, PowerPoint, and scanned images remain mixed together when input into RAG, the accuracy of text extraction will vary by file format, resulting in some documents being found in searches while others with the same content are not. Text-centric documents should be standardized into PDF or plain text, while documents containing many tables and figures should use a format that preserves their structure for stability.

Metadata assignment is easy to neglect, but it actually has a greater impact on search accuracy than file format. Assigning items such as title, department, effective date, confidentiality classification, and document type to each document makes it significantly easier to narrow down search results and display the basis for answers. Since the effective date is directly linked to version management of regulations and manuals discussed later, deciding on input rules at this stage reduces the need for corrections in later processes. It is realistic to limit the number of items to around five without being overly ambitious—adding more items increases the likelihood of missing entries, which becomes a cause of delays in the organization process itself.

For details on text splitting and chunk design, the content covered in How to Build an AI Chatbot for Lao Language Support may also serve as a useful reference.

The process should not end once document preparation is complete — an essential step is to actually feed the documents into RAG and run search tests. Whether the prepared documents can be correctly retrieved by the search engine or AI cannot be known until you actually try asking questions.

The testing method is simple. List around 10 to 20 questions that are commonly asked on-site, and actually input them into the AI to check the answers. Verify one by one whether the expected answer is returned, whether the documents cited as sources are correct, and whether outdated versions or duplicate documents have been mixed in.

When an answer is off the mark, the cause generally falls into one of three patterns. If the document itself lacks the necessary information, additional preparation is needed. If the information exists but isn't being picked up by search, revisiting the metadata or title is effective. If both the information and the search are fine but the answer is still inaccurate, adjustments on the search algorithm side may be necessary.

Among these, improving the accuracy of the search algorithm is a specialized area separate from document preparation. Details on techniques such as hybrid search and reranking are covered in the article on hybrid search for improving search accuracy, so this article will limit itself to confirming whether the document side is properly prepared. Issues uncovered during testing are worth accumulating as an improvement list for when you expand the priority scope, which will prove useful in the next inventory cycle.

The key points to watch for in document preparation differ depending on the type of document. Let's review the pitfalls that are easy to overlook for three types: regulations/manuals, meeting minutes/daily reports, and scanned PDFs.

Regulations and manuals are documents that undergo repeated revisions, so when feeding them into RAG, it is essential to clearly identify "which version is the latest." At first, it might seem sufficient to keep only the latest version in the folder, but in practice, rather than completely deleting older versions, it works better to exclude them from search targets while retaining their effective date and expiration date as metadata. This is because doing otherwise makes it impossible to trace revision history or respond to inquiries seeking to confirm the basis for past operations.

Specifically, it is effective to establish a practice of always including "effective date," "version number," "approver," and "reason for revision" in the file name or metadata. For example, with work regulations, if the AI references a version that does not clearly state its effective date, it could end up answering with a leave system that has already been abolished. When there are provisions whose content contradicts before and after a revision, it is advisable to tag the old version as "not for reference" and physically remove it from the search index.

Additionally, regulations often have their own local rules that vary by department, so tagging to distinguish company-wide regulations from local regulations is also important. By establishing version management rules, you can maintain a state in which the AI always answers based only on currently valid regulations.

Unlike regulations and manuals, meeting minutes and daily reports tend to vary in format and level of detail depending on the author, and there are many cases where personal names, evaluation comments, and client information are recorded as-is. If remarks related to personnel evaluations, or complaints/dissatisfaction about specific clients, remain in meeting minutes, there is a risk that the AI will cite them as search results even for unrelated questions.

When preparing these documents, it is effective to first divide the content into "business decisions" and "personal remarks/evaluations." Decisions have value as knowledge, but records of remarks and personal evaluations often have little need to be fed into RAG, requiring a judgment call to separate them by confidentiality classification and exclude them.

Daily reports similarly tend to mix in notes about an individual's attendance or health condition. When names or client names are included in the body text, consider masking them or replacing them with summaries before input, referring to the approach shown in guidelines on anonymously processed information. The guidelines from the Personal Information Protection Commission stipulate an obligation to disclose the categories of information included when creating information processed so that specific individuals cannot be identified; applying the same principle to internal documents — clearly documenting "which information to keep and which to drop" — helps keep operations consistent.

Since meeting minutes and daily reports are updated frequently but often have low reuse value, a practical approach is not to treat all of them as targets for preparation, but to select only those that could serve as grounds for internal regulations.

Paper-based regulation manuals, PDFs scanned from past application forms, and photos of posted notices may not be recognized as text when fed directly into RAG, often resulting in them not appearing in search results at all. They only become searchable once they have been converted into text using OCR (Optical Character Recognition).

A point often overlooked here is quality verification after OCR processing. OCR tends to misrecognize handwritten characters, tables with many ruled lines, and characters near faded stamps. If misconversions occur—such as reading "日" as "目" or "1" as "l"—the search keywords will not match the text in the document, and the relevant file will not appear in the search results.

As a countermeasure, it is effective to incorporate a process of visually sampling the text after OCR processing to check whether important numerical values, dates, and proper nouns have been converted correctly. Priority should be given to checking sections where errors could impact business decisions, such as amount and date fields in contracts.

For low-resolution PDFs created with old scanning equipment, it may be more efficient to consider re-scanning or, if possible, searching for and replacing them with the original electronic data rather than spending time trying to improve OCR accuracy.

Document maintenance is not a one-time task. To preserve the state achieved immediately after an inventory review, it is essential to have an update owner in place, a regular review cycle, and a mechanism for synchronizing the source documents with the AI-side data. This section explains how to build a system that sustains ongoing operations.

Even after document maintenance is put in order once, if the person in charge changes and updates stop, the state will deteriorate within a few months. What is indispensable as a mechanism for sustaining maintenance is clarifying who is responsible for updates and establishing an inventory review cycle.

The basic principle is to narrow down "who guarantees correctness" to a single person for each document. For regulations, the General Affairs or HR department is appropriate; for operational manuals, the leader of each site is appropriate—in other words, someone who actually uses the document and can grasp changes to it. It is not realistic for the IT department to grasp and update the content of everything, and a division of roles tends to work well: the business side is responsible for the accuracy of content, while the IT side is responsible for the consistency of format and metadata.

It is practical to divide the inventory review cycle according to the update frequency of the documents.

• Documents with infrequent revisions, such as regulations and manuals: inventory review every six months to once a year
• Documents with frequent updates, such as meeting minutes, FAQs, and procedure manuals: checked monthly to quarterly
• Documents involving changes to laws or internal company systems: updated as needed whenever a change occurs

If you operate on a "fix it whenever you notice" basis without setting a cycle, old versions keep lingering and become a cause of the AI answering with outdated information. Recording the results of the inventory review in a simple ledger or checklist, so that the next person in charge can trace the situation, forms the foundation for maintaining the state of order over the long term.

Even if the source documents are updated, if the index that RAG references is not automatically updated, the AI will keep answering with outdated content. The point of discussion in this section is building a synchronization mechanism between the source documents and the search data.

The ideal is a pipeline that detects changes in the document storage location and automatically re-indexes. Configuring the system so that update events from cloud storage or knowledge management systems trigger differential reflection of additions, modifications, and deletions can prevent manual reflection from being missed. If automation is difficult, periodic batch processing according to update frequency can serve as a substitute, but users need to be informed of the time lag until reflection (for example, up to one day for a daily batch).

What requires attention is the synchronization of deletions. If a source document is deleted or unpublished but remains in the search data, the AI may answer based on information that is no longer valid. It is essential to design bidirectional synchronization that covers not only additions and updates but also deletions.

For details on text splitting and index structure design itself, see How to Build an AI Chatbot for the Lao Language — Achieving a Practical Level with a Low-Resource Language × RAG.

In the field of document maintenance, two failures are repeatedly seen: stagnation caused by perfectionism, and deterioration due to loss of synchronization after maintenance is completed. Both can be avoided by slightly changing the approach. The following H3 sections concretely organize the causes and countermeasures.

When the IT department aims for perfect maintenance of all documents, there are cases where several months pass just in the planning stage before work even begins. In organizations where the target documents number in the hundreds or thousands, attempting to complete confidentiality classification and metadata assignment all at once takes so much time adjusting judgment criteria and confirming with related departments that the process never even reaches the search testing stage.

At sites that overcome this obstacle, the judgment to prioritize speed of starting over perfection proves effective. This is an approach of completing the inventory review and confidentiality classification for only a subset of frequently used regulations and manuals—rather than all documents—and proceeding to search testing first. When search is actually tried out, unexpected notational inconsistencies and mixed-in old versions are discovered, which can be directly incorporated into the criteria-building for later stages.

As a guideline: for a small department with only a few dozen target documents, it is possible to proceed with comprehensive maintenance without narrowing the scope. However, when the effort spans multiple departments and exceeds several hundred documents, deciding on a priority area and expanding in stages makes the rollout less likely to stall. Rather than fixing the maintenance plan too rigidly beforehand, the sequence of verifying on a small scale first and then expanding ultimately raises the level of completeness of the maintenance as well.

Even after document preparation has been carried out once, if synchronization with RAG is lost during subsequent operations, search accuracy will gradually deteriorate. A typical case is when the original document is updated, but the index and vector data already ingested into RAG are left outdated. In this case, the AI continues to answer based on the pre-update content as if it were correct.

Situations where synchronization loss tends to occur can be understood more clearly by dividing them into several conditions.

• In operations where documents are manually re-ingested into RAG, if the person in charge forgets to update, a discrepancy arises immediately
• Even when there is a folder integration or automatic crawling mechanism, documents that are moved or saved outside the scope of the target folder will not be ingested
• When documents are deleted or consolidated, only the old version's index may remain, causing duplicate answers

As a countermeasure, rather than an operation where documents are re-ingested every time they are updated, it is effective to align the document storage location with RAG's ingestion target and build in a mechanism that automates the process from update detection to reflection. Text splitting and improvements to search accuracy are separate considerations, and while they also relate to the design decisions covered in Bringing Enterprise RAG into Production, here it is first necessary to keep in mind that the operational practice of regularly confirming "whether the original document and the RAG state are always consistent" is itself the starting point for preventing degradation.

Q1. Where should we start when preparing internal documents? Rather than preparing all documents at once, it is more efficient to begin with areas that have both high usage frequency and high update frequency. Prioritizing frequently-inquired-about regulations and manuals, first completing the inventory and confidentiality classification, and then proceeding to format standardization and metadata assignment tends to be a practically manageable order.

Q2. Can confidential documents not be ingested into RAG? The handling varies depending on the confidentiality classification. Documents containing personal information or HR information need to be considered on the premise of anonymization processing and access control, and it is realistic to either exclude them from the ingestion target or separate them into a dedicated area with limited reference scope. The design of the classification itself should be decided at the inventory stage.

Q3. Should scanned PDFs and OCR documents also be subject to preparation? Documents with low OCR accuracy tend to cause incorrect answers if ingested as-is, so it is necessary to first verify the quality of the text conversion. It is also an option to lower the priority of documents with many reading errors and instead re-digitize the more frequently used documents first.

Q4. How often should prepared documents be reviewed? For regulations and manuals with high update frequency, a mechanism in which the AI's reference data is also updated each time a change occurs is desirable. For documents with low update frequency, it is sufficient to check them in line with the regular inventory cycle; in practice, it makes sense to vary the review frequency depending on the nature of the document.

Q5. Are text splitting and search accuracy improvements covered in this article? This article focuses specifically on document preparation prior to ingestion. Chunk splitting design is explained in detail in How to Build a Lao-Language AI Chatbot — Achieving Practical-Level Performance with Low-Resource Languages × RAG, and methods for improving search accuracy are explained in detail in Bringing Enterprise RAG into Production — Implementation Patterns of Agentic RAG and Hybrid Search Verified with a Lao-Language Chatbot.

Preparing internal documents for AI use is not a one-time, all-encompassing project covering every document. Taking inventory starting from high-usage areas, organizing confidentiality classifications and formats, and expanding the scope while confirming through search testing leads to more stable results, both in terms of time to deployment and the quality of verification.

Rather than delaying the start in pursuit of perfect preparation, it is more practical to prepare only the priority areas, move quickly to search testing, and identify gaps based on how the answers miss the mark. The biggest factor that stalls preparation is the mindset of "doing everything before starting" itself.

After preparation, it is essential to determine an update owner and inventory cycle, and to maintain a synchronization mechanism so that document revisions are reflected in the knowledge base. If synchronization is lost, even well-prepared documents will quickly deteriorate. Technical optimizations after ingestion, such as text splitting and search accuracy improvements, are areas best left to separate specialized procedures depending on the purpose. Starting small, confirming quality through search testing, and gradually expanding the scope and operational structure is the realistic approach to successfully building a knowledge base.