FTSC RAG Search Tool

Guidelines for Use

Please ensure you read and fully acknowledge this disclaimer before utilizing this tool:

  1. This tool is offered solely as a research aid to investigate flight test methodologies, insights, best practices and lessons learned. It is imperative to understand that its outputs are not validated and must be treated as purely informational.
  2. Your engagement with this tool is strictly at your own risk. The Flight Test Safety Committee disclaims all responsibility for any consequences or decisions based on the information provided by this tool.
  3. This tool uses AI that may create fake information (hallucinate) or fabricate sources. Always verify responses carefully.
  4. Conversely, responses from the tool may also be partial, obsolete, or contextually incorrect. The model's database also does not include confidential, classified, or program-specific information.
  5. Results from this tool do not replace regulatory requirements, engineering judgement, official guidance or proven operational procedures.
  6. You use this tool at your own risk and it is highly encouraged for qualified subject matter experts to review all outputs before use in flight test planning or execution.

FTSC Paper Database Search Tool

Search the FTSC paper database using retrieval-augmented generation

Welcome to the Flight Test Safety Committee's paper database RAG search tool! This tool uses Retrieval Augmented Generation, a form of large language model automation.

You can use this tool to search the FTSC paper database for sources about a specific topic.

Guidelines for using the tool:

  1. You can ask in complete sentences ("What do I need to know about high-altitude flight test?") or short phrases ("high-altitude flight testing")
  2. The LLM may create fake information (hallucinate), or even fabricate entire sources. Carefully check all responses.
  3. If the model returns "no relevant information found in the available sources", try submitting a slightly longer, more detailed prompt with more keywords.
  4. LLM performance tends to degrade as a conversation gets longer. We strongly recommend clearing the conversation history using the "Clear Chat" button when you switch topics, or when the conversation starts to get long.

How does this tool work?

This tool uses a technique called Retrieval-Augmented Generation, which uses a large language model (LLM) connected to a database of information.

Retrieval-Augmented Generation (RAG) is a technique to fine-tune an LLM to a specific database or use case without the need for retraining, which would be expensive and infeasible for small-scale use.

When you ask a question, the RAG tool first generates a general answer and then backs it up with information retrieved directly from the database.

RAG does the following:

1. Database Creation: The files in the database are broken into text "chunks" which are converted into numerical vectors ("embeddings") that encode their semantic meaning. From this point on, the RAG tool only uses this vector database of chunk embeddings, and does not have access to the raw files (e.g. PDFs) in the original database.

2. Document Retrieval: When you ask a query, it is converted into a numerical vector embedding in the same way as the database files. The retrieval system then searches through the embeddings of all corpus documents to find the document chunks that are most similar to the query (using vector euclidean distance or another metric). These correspond to the most relevant documents.

3. Context Assembly: The most relevant document chunks are retrieved and combined with your original question and conversation history to create a comprehensive context.

4. Response Generation: A large language model (in our case, a lightweight variant of Gemini) uses the combined context from step 3 to generate a response to your query.

Because the context from step 3 only contains the most relevant content from the database, the LLM's response is tailored to your query and is less likely to be distracted by irrelevant content.

Multi-message conversations: This tool now maintains conversation history. Each new message includes the previous conversation context, allowing for follow-up questions and coherent multi-turn discussions.

Like other LLM-based chat tools (e.g. ChatGPT, Claude, Gemini), this tool uses a system prompt which your query is appended to. This prompt shapes the model's behavior, tone, and things it is allowed and not allowed to say in response to your query.

💡 Example Queries:
What do I need to know about high-altitude flight test?
What does the database say about safety mishap accountability?
Aircraft certification requirements
(Follow up) What are the safety considerations for this?