crack_chunk_embed input_data=/mnt/azureml/cr/j/225599a450e144c18d10782ad261c6cf/cap/data-capability/wd/INPUT_input_data input_glob=**/* chunk_size=1024 chunk_overlap=0 citation_url=None citation_replacement_regex=None custom_loader=None embeddings_model=azure_open_ai://deployment/text-embedding-ada-002/model/text-embedding-ada-002 embeddings_connection_id=/subscriptions/bf4f4af4-8cb7-420c-b682-722abbdff681/resourceGroups/enterpriseit-ai/providers/Microsoft.MachineLearningServices/workspaces/confluence-fdt/connections/kshe-m7llzpu3-eastus2_aoai embeddings_container=None batch_size=100 num_workers=-1 output_path=/mnt/azureml/cr/j/225599a450e144c18d10782ad261c6cf/cap/data-capability/wd/embeddings verbosity=0 doc_intel_connection_id=None max_sample_files=-1 use_rcts=True [2025-02-27 07:19:04] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - ActivityStarted, crack_and_chunk_and_embed (activity.py:108) [2025-02-27 07:19:05] INFO azureml.rag.connections - Getting workspace connection: kshe-m7llzpu3-eastus2_aoai, with input credential: . (connections.py:332) [2025-02-27 07:19:05] INFO azureml.rag.connections - Getting credential from AzureMLTokenAuthentication._initialize_aml_token_auth (connections.py:341) [2025-02-27 07:19:05] INFO azureml.rag.connections - Getting workspace connection via MLClient with auth: , subscription_id: bf4f4af4-8cb7-420c-b682-722abbdff681, resource_group_name: enterpriseit-ai, workspace_name: confluence-fdt. (connections.py:348) Method connections: This is an experimental method, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information. [2025-02-27 07:19:05] INFO azureml.rag.connections - Using ml_client base_url: https://eastus.api.azureml.ms/rp/workspaces, original_base_url: https://management.azure.com. (connections.py:367) [2025-02-27 07:19:05] INFO azureml.rag.connections - Parsed Connection: /subscriptions/bf4f4af4-8cb7-420c-b682-722abbdff681/resourceGroups/enterpriseit-ai/providers/Microsoft.MachineLearningServices/workspaces/confluence-fdt/connections/kshe-m7llzpu3-eastus2_aoai (connections.py:386) [2025-02-27 07:19:05] INFO azureml.rag.connections - Got connection: /subscriptions/bf4f4af4-8cb7-420c-b682-722abbdff681/resourceGroups/enterpriseit-ai/providers/Microsoft.MachineLearningServices/workspaces/confluence-fdt/connections/kshe-m7llzpu3-eastus2_aoai as . (connections.py:443) [2025-02-27 07:19:05] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:19:06] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - ActivityStarted, create_embeddings (activity.py:108) [2025-02-27 07:19:09] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2023-10-13-Disaster-Recovery-Drill-Report_3209625648.html (crack_and_chunk.py:127) [2025-02-27 07:19:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2023-10-13-Disaster-Recovery-Drill-Report_3209625648.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2023-11-17-Disaster-Recovery-Drill-Report_3245342721.html (crack_and_chunk.py:127) [2025-02-27 07:19:17] INFO azureml.rag.chunk_embedder_1 - chunk_embedder_1 started (embed.py:142) [2025-02-27 07:19:17] INFO azureml.rag.chunk_embedder_0 - chunk_embedder_0 started (embed.py:142) [2025-02-27 07:19:17] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:19:17] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:19:17] INFO azureml.rag.chunk_embedder_0.embed.chunk_embedder_0 - ActivityStarted, embed.chunk_embedder_0 (activity.py:108) [2025-02-27 07:19:17] INFO azureml.rag.chunk_embedder_0 - waiting for chunk_batch: 0 (embed.py:159) [2025-02-27 07:19:17] INFO azureml.rag.chunk_embedder_1.embed.chunk_embedder_1 - ActivityStarted, embed.chunk_embedder_1 (activity.py:108) [2025-02-27 07:19:17] INFO azureml.rag.chunk_embedder_1 - waiting for chunk_batch: 1 (embed.py:159) [2025-02-27 07:19:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2023-11-17-Disaster-Recovery-Drill-Report_3245342721.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2023-12-07-Disaster-Recovery-Drill-Report_3284893697.html (crack_and_chunk.py:127) [2025-02-27 07:19:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2023-12-07-Disaster-Recovery-Drill-Report_3284893697.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2024-01-12-Database-Snapshot-Restoration-Report_3278372865.html (crack_and_chunk.py:127) [2025-02-27 07:19:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2024-01-12-Database-Snapshot-Restoration-Report_3278372865.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2024-01-12-Disaster-Recovery-Drill-Report_3285123073.html (crack_and_chunk.py:127) [2025-02-27 07:19:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2024-01-12-Disaster-Recovery-Drill-Report_3285123073.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2024-01-19-Disaster-Recovery-Drill-Report_3320414228.html (crack_and_chunk.py:127) [2025-02-27 07:19:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2024-01-19-Disaster-Recovery-Drill-Report_3320414228.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:19] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2024-Release-Notes-List_3865542657.html (crack_and_chunk.py:127) [2025-02-27 07:19:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2024-Release-Notes-List_3865542657.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2025-Release-Notes-List_3865575440.html (crack_and_chunk.py:127) [2025-02-27 07:19:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:24] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2025-Release-Notes-List_3865575440.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:24] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2875785425.html (crack_and_chunk.py:127) [2025-02-27 07:19:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2875785425.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2968813783.html (crack_and_chunk.py:127) [2025-02-27 07:19:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:29] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2968813783.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:31] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 10/10 processed_sources (logging.py:383) [2025-02-27 07:19:33] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 17/17 documents_total (logging.py:383) [2025-02-27 07:19:35] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:19:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/2968813873.html (crack_and_chunk.py:127) [2025-02-27 07:19:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/2968813873.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3143794697.html (crack_and_chunk.py:127) [2025-02-27 07:19:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3143794697.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3157426183.html (crack_and_chunk.py:127) [2025-02-27 07:19:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:41] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3157426183.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:41] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3198353494.html (crack_and_chunk.py:127) [2025-02-27 07:19:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3198353494.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3228270695.html (crack_and_chunk.py:127) [2025-02-27 07:19:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:46] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3228270695.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3243573298.html (crack_and_chunk.py:127) [2025-02-27 07:19:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3243573298.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3254059108.html (crack_and_chunk.py:127) [2025-02-27 07:19:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3254059108.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:49] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3266379876.html (crack_and_chunk.py:127) [2025-02-27 07:19:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:51] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3266379876.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:51] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3266412547.html (crack_and_chunk.py:127) [2025-02-27 07:19:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3266412547.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3277586526.html (crack_and_chunk.py:127) [2025-02-27 07:19:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3277586526.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:19:55] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 20/20 processed_sources (logging.py:383) [2025-02-27 07:19:56] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 28/28 documents_total (logging.py:383) [2025-02-27 07:19:59] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:19:59] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3284795478.html (crack_and_chunk.py:127) [2025-02-27 07:19:59] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:19:59] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3284795478.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3306324099.html (crack_and_chunk.py:127) [2025-02-27 07:20:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3306324099.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3311337543.html (crack_and_chunk.py:127) [2025-02-27 07:20:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:02] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3311337543.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:02] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3311435908.html (crack_and_chunk.py:127) [2025-02-27 07:20:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3311435908.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3311436016.html (crack_and_chunk.py:127) [2025-02-27 07:20:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3311436016.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:04] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3317137424.html (crack_and_chunk.py:127) [2025-02-27 07:20:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3317137424.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3319169047.html (crack_and_chunk.py:127) [2025-02-27 07:20:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:08] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3319169047.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3340042242.html (crack_and_chunk.py:127) [2025-02-27 07:20:09] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:09] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:11] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3340042242.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:11] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3415113733.html (crack_and_chunk.py:127) [2025-02-27 07:20:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:12] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3415113733.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3421929525.html (crack_and_chunk.py:127) [2025-02-27 07:20:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:14] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3421929525.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:16] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 30/30 processed_sources (logging.py:383) [2025-02-27 07:20:17] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 40/40 documents_total (logging.py:383) [2025-02-27 07:20:18] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:20:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3440738316.html (crack_and_chunk.py:127) [2025-02-27 07:20:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3440738316.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3446243340.html (crack_and_chunk.py:127) [2025-02-27 07:20:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3446243340.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3452567628.html (crack_and_chunk.py:127) [2025-02-27 07:20:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3452567628.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3457450006.html (crack_and_chunk.py:127) [2025-02-27 07:20:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3457450006.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3496017921.html (crack_and_chunk.py:127) [2025-02-27 07:20:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3496017921.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3497361416.html (crack_and_chunk.py:127) [2025-02-27 07:20:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3497361416.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3501391905.html (crack_and_chunk.py:127) [2025-02-27 07:20:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3501391905.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3502178306.html (crack_and_chunk.py:127) [2025-02-27 07:20:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3502178306.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3504078861.html (crack_and_chunk.py:127) [2025-02-27 07:20:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3504078861.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3504701441.html (crack_and_chunk.py:127) [2025-02-27 07:20:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3504701441.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:32] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 40/40 processed_sources (logging.py:383) [2025-02-27 07:20:34] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 52/52 documents_total (logging.py:383) [2025-02-27 07:20:35] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:20:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3511320584.html (crack_and_chunk.py:127) [2025-02-27 07:20:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:37] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3511320584.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:37] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3517022210.html (crack_and_chunk.py:127) [2025-02-27 07:20:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3517022210.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3517022244.html (crack_and_chunk.py:127) [2025-02-27 07:20:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3517022244.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3517153311.html (crack_and_chunk.py:127) [2025-02-27 07:20:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:41] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3517153311.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:41] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3517808642.html (crack_and_chunk.py:127) [2025-02-27 07:20:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3517808642.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3522888199.html (crack_and_chunk.py:127) [2025-02-27 07:20:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:43] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3522888199.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:43] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3529900042.html (crack_and_chunk.py:127) [2025-02-27 07:20:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:45] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3529900042.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:45] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3542155296.html (crack_and_chunk.py:127) [2025-02-27 07:20:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3542155296.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3547201549.html (crack_and_chunk.py:127) [2025-02-27 07:20:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:48] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3547201549.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:48] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3549298691.html (crack_and_chunk.py:127) [2025-02-27 07:20:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3549298691.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:50] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 50/50 processed_sources (logging.py:383) [2025-02-27 07:20:52] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 62/62 documents_total (logging.py:383) [2025-02-27 07:20:54] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:20:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3550904331.html (crack_and_chunk.py:127) [2025-02-27 07:20:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3550904331.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3554082866.html (crack_and_chunk.py:127) [2025-02-27 07:20:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:58] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3554082866.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:20:58] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3554115639.html (crack_and_chunk.py:127) [2025-02-27 07:20:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:20:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3554115639.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3554246658.html (crack_and_chunk.py:127) [2025-02-27 07:21:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:01] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3554246658.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:01] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3554312193.html (crack_and_chunk.py:127) [2025-02-27 07:21:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3554312193.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3554312194.html (crack_and_chunk.py:127) [2025-02-27 07:21:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3554312194.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3583410178.html (crack_and_chunk.py:127) [2025-02-27 07:21:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3583410178.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3592650761.html (crack_and_chunk.py:127) [2025-02-27 07:21:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:08] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3592650761.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3600711683.html (crack_and_chunk.py:127) [2025-02-27 07:21:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:09] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3600711683.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:09] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3606544385.html (crack_and_chunk.py:127) [2025-02-27 07:21:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:11] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3606544385.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:13] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 60/60 processed_sources (logging.py:383) [2025-02-27 07:21:15] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 75/75 documents_total (logging.py:383) [2025-02-27 07:21:16] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:21:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3628171266.html (crack_and_chunk.py:127) [2025-02-27 07:21:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3628171266.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3636002818.html (crack_and_chunk.py:127) [2025-02-27 07:21:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3636002818.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3636461581.html (crack_and_chunk.py:127) [2025-02-27 07:21:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3636461581.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3646947330.html (crack_and_chunk.py:127) [2025-02-27 07:21:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3646947330.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3661758465.html (crack_and_chunk.py:127) [2025-02-27 07:21:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3661758465.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3668803715.html (crack_and_chunk.py:127) [2025-02-27 07:21:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:26] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3668803715.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:26] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3668902028.html (crack_and_chunk.py:127) [2025-02-27 07:21:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3668902028.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3669753863.html (crack_and_chunk.py:127) [2025-02-27 07:21:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3669753863.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3669786628.html (crack_and_chunk.py:127) [2025-02-27 07:21:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:29] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3669786628.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:29] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3669983235.html (crack_and_chunk.py:127) [2025-02-27 07:21:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3669983235.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:33] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 70/70 processed_sources (logging.py:383) [2025-02-27 07:21:35] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 85/85 documents_total (logging.py:383) [2025-02-27 07:21:38] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:21:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3680927804.html (crack_and_chunk.py:127) [2025-02-27 07:21:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3680927804.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3692986370.html (crack_and_chunk.py:127) [2025-02-27 07:21:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3692986370.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3702161426.html (crack_and_chunk.py:127) [2025-02-27 07:21:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:41] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3702161426.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:41] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3706224644.html (crack_and_chunk.py:127) [2025-02-27 07:21:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3706224644.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3717922817.html (crack_and_chunk.py:127) [2025-02-27 07:21:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3717922817.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3723952129.html (crack_and_chunk.py:127) [2025-02-27 07:21:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:46] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3723952129.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3726082062.html (crack_and_chunk.py:127) [2025-02-27 07:21:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3726082062.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3744169986.html (crack_and_chunk.py:127) [2025-02-27 07:21:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3744169986.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:49] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3746299922.html (crack_and_chunk.py:127) [2025-02-27 07:21:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3746299922.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3761438721.html (crack_and_chunk.py:127) [2025-02-27 07:21:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:51] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3761438721.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:53] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 80/80 processed_sources (logging.py:383) [2025-02-27 07:21:55] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 99/99 documents_total (logging.py:383) [2025-02-27 07:21:55] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:21:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3765534733.html (crack_and_chunk.py:127) [2025-02-27 07:21:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:21:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3765534733.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:21:56] INFO azureml.rag.embed - ==== Putting batch_id=0 with 100 chunks in queue (embed.py:320) [2025-02-27 07:21:59] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3769237670.html (crack_and_chunk.py:127) [2025-02-27 07:21:59] INFO azureml.rag.chunk_embedder_0 - ==== embedding batch_id=0 with 100 chunks (embed.py:166) [2025-02-27 07:21:59] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2023-10-13-Disaster-Recovery-Drill-Report_3209625648.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2023-10-13-Disaster-Recovery-Drill-Report_3209625648.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2023-10-13-Disaster-Recovery-Drill-Report_3209625648.html2 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2023-11-17-Disaster-Recovery-Drill-Report_3245342721.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2023-11-17-Disaster-Recovery-Drill-Report_3245342721.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2023-12-07-Disaster-Recovery-Drill-Report_3284893697.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2023-12-07-Disaster-Recovery-Drill-Report_3284893697.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2024-01-12-Database-Snapshot-Restoration-Report_3278372865.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2024-01-12-Database-Snapshot-Restoration-Report_3278372865.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2024-01-12-Disaster-Recovery-Drill-Report_3285123073.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2024-01-12-Disaster-Recovery-Drill-Report_3285123073.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2024-01-19-Disaster-Recovery-Drill-Report_3320414228.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2024-01-19-Disaster-Recovery-Drill-Report_3320414228.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2024-Release-Notes-List_3865542657.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2025-Release-Notes-List_3865575440.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2875785425.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2968813783.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/2968813873.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3143794697.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3157426183.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3198353494.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3228270695.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3243573298.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3243573298.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3254059108.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3266379876.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3266412547.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3277586526.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3284795478.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3306324099.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3306324099.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3306324099.html2 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3311337543.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3311435908.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3311436016.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3317137424.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3319169047.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3340042242.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3415113733.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3421929525.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3440738316.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3440738316.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3446243340.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3452567628.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3457450006.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3496017921.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3496017921.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3497361416.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3501391905.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3502178306.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3504078861.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3504701441.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3511320584.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3517022210.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3517022244.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3517153311.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3517808642.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3522888199.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3529900042.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3542155296.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3547201549.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3549298691.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3550904331.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3554082866.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3554115639.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3554246658.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3554312193.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3554312194.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3583410178.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3583410178.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3583410178.html2 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3583410178.html3 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3592650761.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3600711683.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3606544385.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3628171266.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3636002818.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3636461581.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3646947330.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3661758465.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3668803715.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3668902028.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3669753863.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3669786628.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3669983235.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3680927804.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3680927804.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3680927804.html2 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3680927804.html3 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3692986370.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3702161426.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3706224644.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3717922817.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3723952129.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3726082062.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3744169986.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3746299922.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3746299922.html1 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3761438721.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3765534733.html0 (__init__.py:1300) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings - Documents to embed: 100 Documents reused: 0 (__init__.py:1337) [2025-02-27 07:22:00] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityStarted, Embeddings.embed (activity.py:108) [2025-02-27 07:22:00] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:22:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3769237670.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3770384386.html (crack_and_chunk.py:127) [2025-02-27 07:22:01] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:22:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:01] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:22:01] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:22:02] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:22:02] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:22:02] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 4 documents. (openai.py:196) [2025-02-27 07:22:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3770384386.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3772973100.html (crack_and_chunk.py:127) [2025-02-27 07:22:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3772973100.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3773005858.html (crack_and_chunk.py:127) [2025-02-27 07:22:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3773005858.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3784146946.html (crack_and_chunk.py:127) [2025-02-27 07:22:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:08] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3784146946.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3784409098.html (crack_and_chunk.py:127) [2025-02-27 07:22:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:09] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3784409098.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:09] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3786801160.html (crack_and_chunk.py:127) [2025-02-27 07:22:09] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:09] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3786801160.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3792961557.html (crack_and_chunk.py:127) [2025-02-27 07:22:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:12] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3792961557.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3793420289.html (crack_and_chunk.py:127) [2025-02-27 07:22:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:13] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3793420289.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:15] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 90/90 processed_sources (logging.py:383) [2025-02-27 07:22:16] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 109/109 documents_total (logging.py:383) [2025-02-27 07:22:17] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:22:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3793453058.html (crack_and_chunk.py:127) [2025-02-27 07:22:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3793453058.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:19] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3793584297.html (crack_and_chunk.py:127) [2025-02-27 07:22:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3793584297.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:19] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3793911907.html (crack_and_chunk.py:127) [2025-02-27 07:22:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:21] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3793911907.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:21] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3793911936.html (crack_and_chunk.py:127) [2025-02-27 07:22:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3793911936.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3794436097.html (crack_and_chunk.py:127) [2025-02-27 07:22:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:24] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3794436097.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:24] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3796402212.html (crack_and_chunk.py:127) [2025-02-27 07:22:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3796402212.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3797090326.html (crack_and_chunk.py:127) [2025-02-27 07:22:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3797090326.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3798630422.html (crack_and_chunk.py:127) [2025-02-27 07:22:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:26] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3798630422.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:26] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3805839361.html (crack_and_chunk.py:127) [2025-02-27 07:22:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3805839361.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3809411082.html (crack_and_chunk.py:127) [2025-02-27 07:22:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3809411082.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:30] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 100/100 processed_sources (logging.py:383) [2025-02-27 07:22:32] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 123/123 documents_total (logging.py:383) [2025-02-27 07:22:42] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:22:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3812294665.html (crack_and_chunk.py:127) [2025-02-27 07:22:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3812294665.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3825205249.html (crack_and_chunk.py:127) [2025-02-27 07:22:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3825205249.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:49] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3835133966.html (crack_and_chunk.py:127) [2025-02-27 07:22:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3835133966.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3837755423.html (crack_and_chunk.py:127) [2025-02-27 07:22:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3837755423.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3847946242.html (crack_and_chunk.py:127) [2025-02-27 07:22:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:53] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3847946242.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:53] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3865542696.html (crack_and_chunk.py:127) [2025-02-27 07:22:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:55] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3865542696.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3865739265.html (crack_and_chunk.py:127) [2025-02-27 07:22:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3865739265.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3877601283.html (crack_and_chunk.py:127) [2025-02-27 07:22:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:58] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3877601283.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:22:58] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3880452097.html (crack_and_chunk.py:127) [2025-02-27 07:22:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:22:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3880452097.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/3891265616.html (crack_and_chunk.py:127) [2025-02-27 07:23:01] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityCompleted: Activity=Embeddings.embed, HowEnded=Success, Duration=60667.65 [ms] (activity.py:129) [2025-02-27 07:23:01] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:23:01] INFO azureml.rag.chunk_embedder_0 - Embedding took 61.25780630111694 seconds (embed.py:184) [2025-02-27 07:23:01] INFO azureml.rag.chunk_embedder_0 - Metadata will be saved (embed.py:198) [2025-02-27 07:23:01] INFO azureml.rag.chunk_embedder_0 - waiting for chunk_batch: 0 (embed.py:159) [2025-02-27 07:23:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:02] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/3891265616.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:03] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 110/110 processed_sources (logging.py:383) [2025-02-27 07:23:05] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 134/134 documents_total (logging.py:383) [2025-02-27 07:23:05] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:23:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AI-Code-of-Conduct_3685941271.html (crack_and_chunk.py:127) [2025-02-27 07:23:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AI-Code-of-Conduct_3685941271.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/API-Security-Design-Considerations_3600547842.html (crack_and_chunk.py:127) [2025-02-27 07:23:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/API-Security-Design-Considerations_3600547842.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:07] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Alarms-and-Event-Rules-Documentation_3172270116.html (crack_and_chunk.py:127) [2025-02-27 07:23:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:08] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Alarms-and-Event-Rules-Documentation_3172270116.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Antivirus-for-AWS-Storage_3528687617.html (crack_and_chunk.py:127) [2025-02-27 07:23:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:09] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Antivirus-for-AWS-Storage_3528687617.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:09] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-AppConfig_3519742030.html (crack_and_chunk.py:127) [2025-02-27 07:23:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:11] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-AppConfig_3519742030.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:11] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Architecture-Diagram_3580461150.html (crack_and_chunk.py:127) [2025-02-27 07:23:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:13] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Architecture-Diagram_3580461150.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:13] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Backup-Report-2024_3702489124.html (crack_and_chunk.py:127) [2025-02-27 07:23:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:14] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Backup-Report-2024_3702489124.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:14] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Backup-Report-2025_3810426901.html (crack_and_chunk.py:127) [2025-02-27 07:23:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Backup-Report-2025_3810426901.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Backup-Report-Design_3702096034.html (crack_and_chunk.py:127) [2025-02-27 07:23:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:16] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Backup-Report-Design_3702096034.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Backup-Report_3702587410.html (crack_and_chunk.py:127) [2025-02-27 07:23:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Backup-Report_3702587410.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:19] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 120/120 processed_sources (logging.py:383) [2025-02-27 07:23:19] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 148/148 documents_total (logging.py:383) [2025-02-27 07:23:21] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:23:21] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Backup-for-TimesTrust-Account_3786342422.html (crack_and_chunk.py:127) [2025-02-27 07:23:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Backup-for-TimesTrust-Account_3786342422.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Billing-Conductor_3693740040.html (crack_and_chunk.py:127) [2025-02-27 07:23:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Billing-Conductor_3693740040.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Chatbot-Client---MS-Teams-Configure_3568369672.html (crack_and_chunk.py:127) [2025-02-27 07:23:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:26] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Chatbot-Client---MS-Teams-Configure_3568369672.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:26] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Cloud_3519348803.html (crack_and_chunk.py:127) [2025-02-27 07:23:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Cloud_3519348803.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Cloudfront-Application-Load-Balancer-Restrict-Access-with-VPC-origins_3754360835.html (crack_and_chunk.py:127) [2025-02-27 07:23:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Cloudfront-Application-Load-Balancer-Restrict-Access-with-VPC-origins_3754360835.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:31] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Config-Action-Notification_3748299110.html (crack_and_chunk.py:127) [2025-02-27 07:23:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Config-Action-Notification_3748299110.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:32] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Configure-SSO-Guide-Step_3586982013.html (crack_and_chunk.py:127) [2025-02-27 07:23:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:33] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Configure-SSO-Guide-Step_3586982013.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:33] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-ECR-Lifecycle-Policy-Get-Remove-Older-Images_3456139281.html (crack_and_chunk.py:127) [2025-02-27 07:23:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:35] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-ECR-Lifecycle-Policy-Get-Remove-Older-Images_3456139281.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Guard-Duty_3504865281.html (crack_and_chunk.py:127) [2025-02-27 07:23:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:36] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Guard-Duty_3504865281.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:36] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-GuardDuty-Malware-Protection-for-EC2_3545169943.html (crack_and_chunk.py:127) [2025-02-27 07:23:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-GuardDuty-Malware-Protection-for-EC2_3545169943.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:40] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 130/130 processed_sources (logging.py:383) [2025-02-27 07:23:41] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 158/158 documents_total (logging.py:383) [2025-02-27 07:23:42] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:23:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-GuardDuty-and-AWS-Inspector-Comparison_3504537610.html (crack_and_chunk.py:127) [2025-02-27 07:23:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-GuardDuty-and-AWS-Inspector-Comparison_3504537610.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Guide-Document-Step-Configure_3429105682.html (crack_and_chunk.py:127) [2025-02-27 07:23:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:45] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Guide-Document-Step-Configure_3429105682.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:45] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-IAM-Identity-Center-Session-Duration_3584655375.html (crack_and_chunk.py:127) [2025-02-27 07:23:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:46] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-IAM-Identity-Center-Session-Duration_3584655375.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-IAM-Permission-for-Binance-DevOps-Team_3681353734.html (crack_and_chunk.py:127) [2025-02-27 07:23:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-IAM-Permission-for-Binance-DevOps-Team_3681353734.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Identity-Center-Integration-Azure-SSO_3532292145.html (crack_and_chunk.py:127) [2025-02-27 07:23:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Identity-Center-Integration-Azure-SSO_3532292145.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Infra-Diagram_3526295560.html (crack_and_chunk.py:127) [2025-02-27 07:23:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:51] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Infra-Diagram_3526295560.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:51] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Infrastructure_3180036097.html (crack_and_chunk.py:127) [2025-02-27 07:23:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Infrastructure_3180036097.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Inspector---CIS-Scan-Configure_3580067929.html (crack_and_chunk.py:127) [2025-02-27 07:23:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:53] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Inspector---CIS-Scan-Configure_3580067929.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:53] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Inspector-vs-Snyk-Comparison_3828645890.html (crack_and_chunk.py:127) [2025-02-27 07:23:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Inspector-vs-Snyk-Comparison_3828645890.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Key-Management-Service_3519774773.html (crack_and_chunk.py:127) [2025-02-27 07:23:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:55] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Key-Management-Service_3519774773.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:23:55] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 140/140 processed_sources (logging.py:383) [2025-02-27 07:23:57] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 170/170 documents_total (logging.py:383) [2025-02-27 07:23:58] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:23:58] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-PRD-Network_3549429852.html (crack_and_chunk.py:127) [2025-02-27 07:23:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:23:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-PRD-Network_3549429852.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Private-Repositories-Images-Responsibility-Person-In-Charges_3485564952.html (crack_and_chunk.py:127) [2025-02-27 07:24:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Private-Repositories-Images-Responsibility-Person-In-Charges_3485564952.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Programmatic-Access-Key-with-Application_3533045765.html (crack_and_chunk.py:127) [2025-02-27 07:24:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:02] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Programmatic-Access-Key-with-Application_3533045765.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:02] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Programmatic-Access_3532881956.html (crack_and_chunk.py:127) [2025-02-27 07:24:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Programmatic-Access_3532881956.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-RDS-R7G-New-Memory-Optimized-Type_3771072919.html (crack_and_chunk.py:127) [2025-02-27 07:24:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-RDS-R7G-New-Memory-Optimized-Type_3771072919.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Resource-Explorer-FAQs_3641802753.html (crack_and_chunk.py:127) [2025-02-27 07:24:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Resource-Explorer-FAQs_3641802753.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Resource-Explorer_3638820867.html (crack_and_chunk.py:127) [2025-02-27 07:24:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Resource-Explorer_3638820867.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:07] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-SSO-Key-Connect_3556507751.html (crack_and_chunk.py:127) [2025-02-27 07:24:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:09] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-SSO-Key-Connect_3556507751.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:09] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-SSO-With-Azure-AD_3532357725.html (crack_and_chunk.py:127) [2025-02-27 07:24:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:11] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-SSO-With-Azure-AD_3532357725.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:11] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-STG-Network_3549462607.html (crack_and_chunk.py:127) [2025-02-27 07:24:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:13] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-STG-Network_3549462607.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:14] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 150/150 processed_sources (logging.py:383) [2025-02-27 07:24:15] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 185/185 documents_total (logging.py:383) [2025-02-27 07:24:16] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:24:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Secret-Manager-Backup_3841622049.html (crack_and_chunk.py:127) [2025-02-27 07:24:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Secret-Manager-Backup_3841622049.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Security-Hub-Integration-to-Jira-Cloud_3525738588.html (crack_and_chunk.py:127) [2025-02-27 07:24:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Security-Hub-Integration-to-Jira-Cloud_3525738588.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Site-to-Site-VPN-Configure-with-FortiGate_3450831004.html (crack_and_chunk.py:127) [2025-02-27 07:24:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Site-to-Site-VPN-Configure-with-FortiGate_3450831004.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-UAT-Network_3549266028.html (crack_and_chunk.py:127) [2025-02-27 07:24:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-UAT-Network_3549266028.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Unidentified-Actions-Alerts-Notification_3541598209.html (crack_and_chunk.py:127) [2025-02-27 07:24:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Unidentified-Actions-Alerts-Notification_3541598209.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Unidentified-Actions-Alerts-for-User-Login-and-S3-Bucket-Trying-Download_3540877359.html (crack_and_chunk.py:127) [2025-02-27 07:24:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:24] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Unidentified-Actions-Alerts-for-User-Login-and-S3-Bucket-Trying-Download_3540877359.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:24] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-Unidentified-Actions-Alerts_3549462585.html (crack_and_chunk.py:127) [2025-02-27 07:24:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-Unidentified-Actions-Alerts_3549462585.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-WAF---Managed-Rule-Group_3557785601.html (crack_and_chunk.py:127) [2025-02-27 07:24:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:26] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-WAF---Managed-Rule-Group_3557785601.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:26] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-WAF-Baseline-rule-groups_3561816079.html (crack_and_chunk.py:127) [2025-02-27 07:24:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-WAF-Baseline-rule-groups_3561816079.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:27] INFO azureml.rag.embed - ==== Putting batch_id=1 with 100 chunks in queue (embed.py:320) [2025-02-27 07:24:32] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Embedded Documents - 100/200 documents_embedded (logging.py:383) [2025-02-27 07:24:33] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Total Documents - 100/200 documents_total (logging.py:383) [2025-02-27 07:24:33] INFO azureml.rag.chunk_embedder_1 - ==== embedding batch_id=1 with 100 chunks (embed.py:166) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3769237670.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-WAF-Free-Rule-Group_3519217734.html (crack_and_chunk.py:127) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3770384386.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3772973100.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3773005858.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3784146946.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3784409098.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3786801160.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3792961557.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3793420289.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3793453058.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3793584297.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3793584297.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3793911907.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3793911936.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3794436097.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3796402212.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3796402212.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3797090326.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3797090326.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3798630422.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3805839361.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3809411082.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3809411082.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3812294665.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3825205249.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3835133966.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3835133966.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3837755423.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3847946242.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3865542696.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3865739265.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3877601283.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3880452097.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/3891265616.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AI-Code-of-Conduct_3685941271.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AI-Code-of-Conduct_3685941271.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/API-Security-Design-Considerations_3600547842.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Alarms-and-Event-Rules-Documentation_3172270116.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Alarms-and-Event-Rules-Documentation_3172270116.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Alarms-and-Event-Rules-Documentation_3172270116.html2 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Alarms-and-Event-Rules-Documentation_3172270116.html3 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Antivirus-for-AWS-Storage_3528687617.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-AppConfig_3519742030.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Architecture-Diagram_3580461150.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Backup-Report-2024_3702489124.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Backup-Report-2025_3810426901.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Backup-Report-Design_3702096034.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Backup-Report_3702587410.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Backup-for-TimesTrust-Account_3786342422.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Billing-Conductor_3693740040.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Chatbot-Client---MS-Teams-Configure_3568369672.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Cloud_3519348803.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Cloudfront-Application-Load-Balancer-Restrict-Access-with-VPC-origins_3754360835.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Config-Action-Notification_3748299110.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Configure-SSO-Guide-Step_3586982013.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-ECR-Lifecycle-Policy-Get-Remove-Older-Images_3456139281.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Guard-Duty_3504865281.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-GuardDuty-Malware-Protection-for-EC2_3545169943.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-GuardDuty-and-AWS-Inspector-Comparison_3504537610.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Guide-Document-Step-Configure_3429105682.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-IAM-Identity-Center-Session-Duration_3584655375.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-IAM-Permission-for-Binance-DevOps-Team_3681353734.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Identity-Center-Integration-Azure-SSO_3532292145.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Infra-Diagram_3526295560.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Infrastructure_3180036097.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Inspector---CIS-Scan-Configure_3580067929.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Inspector-vs-Snyk-Comparison_3828645890.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Inspector-vs-Snyk-Comparison_3828645890.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Inspector-vs-Snyk-Comparison_3828645890.html2 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Key-Management-Service_3519774773.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-PRD-Network_3549429852.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Private-Repositories-Images-Responsibility-Person-In-Charges_3485564952.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Private-Repositories-Images-Responsibility-Person-In-Charges_3485564952.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Private-Repositories-Images-Responsibility-Person-In-Charges_3485564952.html2 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Programmatic-Access-Key-with-Application_3533045765.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Programmatic-Access_3532881956.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-RDS-R7G-New-Memory-Optimized-Type_3771072919.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Resource-Explorer-FAQs_3641802753.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Resource-Explorer-FAQs_3641802753.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Resource-Explorer_3638820867.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Resource-Explorer_3638820867.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Resource-Explorer_3638820867.html2 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-SSO-Key-Connect_3556507751.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-SSO-With-Azure-AD_3532357725.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-STG-Network_3549462607.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Secret-Manager-Backup_3841622049.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Security-Hub-Integration-to-Jira-Cloud_3525738588.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Site-to-Site-VPN-Configure-with-FortiGate_3450831004.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Site-to-Site-VPN-Configure-with-FortiGate_3450831004.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-UAT-Network_3549266028.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Unidentified-Actions-Alerts-Notification_3541598209.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Unidentified-Actions-Alerts-for-User-Login-and-S3-Bucket-Trying-Download_3540877359.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Unidentified-Actions-Alerts-for-User-Login-and-S3-Bucket-Trying-Download_3540877359.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-Unidentified-Actions-Alerts_3549462585.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF---Managed-Rule-Group_3557785601.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF-Baseline-rule-groups_3561816079.html0 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF-Baseline-rule-groups_3561816079.html1 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF-Baseline-rule-groups_3561816079.html2 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF-Baseline-rule-groups_3561816079.html3 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF-Baseline-rule-groups_3561816079.html4 (__init__.py:1300) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings - Documents to embed: 100 Documents reused: 0 (__init__.py:1337) [2025-02-27 07:24:33] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityStarted, Embeddings.embed (activity.py:108) [2025-02-27 07:24:34] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:24:34] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:24:34] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:24:35] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-WAF-Free-Rule-Group_3519217734.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:35] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:24:35] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:24:36] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:24:36] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 160/160 processed_sources (logging.py:383) [2025-02-27 07:24:37] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 201/201 documents_total (logging.py:383) [2025-02-27 07:24:38] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:24:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AWS-WAF_3519348850.html (crack_and_chunk.py:127) [2025-02-27 07:24:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AWS-WAF_3519348850.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:39] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Access-Management-for-FDT-Fortinet-Firewalls._3281256454.html (crack_and_chunk.py:127) [2025-02-27 07:24:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Access-Management-for-FDT-Fortinet-Firewalls._3281256454.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Access-Management_2880634881.html (crack_and_chunk.py:127) [2025-02-27 07:24:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Access-Management_2880634881.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Activate-Bitwarden-account_3161751583.html (crack_and_chunk.py:127) [2025-02-27 07:24:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Activate-Bitwarden-account_3161751583.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Add-FD121-Email-Account-in-Windows_3848994817.html (crack_and_chunk.py:127) [2025-02-27 07:24:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:45] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Add-FD121-Email-Account-in-Windows_3848994817.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:45] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Adding-FDT-Company-Holidays-Calendar-to-Your-Outlook_3298394485.html (crack_and_chunk.py:127) [2025-02-27 07:24:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:48] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Adding-FDT-Company-Holidays-Calendar-to-Your-Outlook_3298394485.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:48] INFO azureml.rag.crack_and_chunk - Processing file: TRM/All-EC2-IMDSv2-Upgrade-for-TimesTrust-Account_3786801229.html (crack_and_chunk.py:127) [2025-02-27 07:24:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/All-EC2-IMDSv2-Upgrade-for-TimesTrust-Account_3786801229.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/All-Sub-Page-Missing-Header-Image_3790667798.html (crack_and_chunk.py:127) [2025-02-27 07:24:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/All-Sub-Page-Missing-Header-Image_3790667798.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html (crack_and_chunk.py:127) [2025-02-27 07:24:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:51] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:51] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Amazon-Q---Integration-with-Confluence_3797614646.html (crack_and_chunk.py:127) [2025-02-27 07:24:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:53] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Amazon-Q---Integration-with-Confluence_3797614646.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:54] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 170/170 processed_sources (logging.py:383) [2025-02-27 07:24:55] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 222/222 documents_total (logging.py:383) [2025-02-27 07:24:55] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:24:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Application-Flow_3526623233.html (crack_and_chunk.py:127) [2025-02-27 07:24:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:57] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Application-Flow_3526623233.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:57] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Application-List_3483041818.html (crack_and_chunk.py:127) [2025-02-27 07:24:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:24:59] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Application-List_3483041818.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:24:59] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Architecture-of-the-External-Database-Backup-Pipeline_3210018832.html (crack_and_chunk.py:127) [2025-02-27 07:25:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Architecture-of-the-External-Database-Backup-Pipeline_3210018832.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Area-1-Email-Security-Enhances_3798007850.html (crack_and_chunk.py:127) [2025-02-27 07:25:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:01] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Area-1-Email-Security-Enhances_3798007850.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:01] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Argo-Smart-Routing_3776839735.html (crack_and_chunk.py:127) [2025-02-27 07:25:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Argo-Smart-Routing_3776839735.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:04] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Assign-Foxit-PDF-Editor-Perpetual-License-to-FDT-Staff_3407347791.html (crack_and_chunk.py:127) [2025-02-27 07:25:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Assign-Foxit-PDF-Editor-Perpetual-License-to-FDT-Staff_3407347791.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Atlassian-Confluence-Cloud-Behind-Firewall_3769466882.html (crack_and_chunk.py:127) [2025-02-27 07:25:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Atlassian-Confluence-Cloud-Behind-Firewall_3769466882.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Atlassian-Intelligence-features-in-Confluence_3797549060.html (crack_and_chunk.py:127) [2025-02-27 07:25:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:08] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Atlassian-Intelligence-features-in-Confluence_3797549060.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Atlassian_3574005810.html (crack_and_chunk.py:127) [2025-02-27 07:25:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Atlassian_3574005810.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Azure-OpenAI-Service_3559063652.html (crack_and_chunk.py:127) [2025-02-27 07:25:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:11] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Azure-OpenAI-Service_3559063652.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:12] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 180/180 processed_sources (logging.py:383) [2025-02-27 07:25:14] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 233/233 documents_total (logging.py:383) [2025-02-27 07:25:14] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:25:14] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Azure-OpenAI-vs-ChatGPT_3549298881.html (crack_and_chunk.py:127) [2025-02-27 07:25:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Azure-OpenAI-vs-ChatGPT_3549298881.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Azure-OpenAI_3572334655.html (crack_and_chunk.py:127) [2025-02-27 07:25:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:16] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Azure-OpenAI_3572334655.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/BYOD-MacOS-Enrollment-Guide_3296788497.html (crack_and_chunk.py:127) [2025-02-27 07:25:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/BYOD-MacOS-Enrollment-Guide_3296788497.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/BYOD-MacOS-Unenroll-Device-Guide_3796107417.html (crack_and_chunk.py:127) [2025-02-27 07:25:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/BYOD-MacOS-Unenroll-Device-Guide_3796107417.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/BYOD-Windows-Enrollment-Guide_3297181721.html (crack_and_chunk.py:127) [2025-02-27 07:25:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:21] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/BYOD-Windows-Enrollment-Guide_3297181721.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:21] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Basic-WAF-Web-ACL-for-TimesTrust-Account_3787161632.html (crack_and_chunk.py:127) [2025-02-27 07:25:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Basic-WAF-Web-ACL-for-TimesTrust-Account_3787161632.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bastion-Server-Maintenance-for-TimesTrust_3786932280.html (crack_and_chunk.py:127) [2025-02-27 07:25:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bastion-Server-Maintenance-for-TimesTrust_3786932280.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Best-Practices-for-Keeping-Your-Google-Account-Secure_3796860955.html (crack_and_chunk.py:127) [2025-02-27 07:25:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Best-Practices-for-Keeping-Your-Google-Account-Secure_3796860955.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Billing-Profile-Separate_3891429378.html (crack_and_chunk.py:127) [2025-02-27 07:25:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Billing-Profile-Separate_3891429378.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Binance-AWS-Project_3681222658.html (crack_and_chunk.py:127) [2025-02-27 07:25:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:29] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Binance-AWS-Project_3681222658.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:30] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 190/190 processed_sources (logging.py:383) [2025-02-27 07:25:31] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 246/246 documents_total (logging.py:383) [2025-02-27 07:25:33] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:25:33] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Binance-Architecture-Diagram_3748397105.html (crack_and_chunk.py:127) [2025-02-27 07:25:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:34] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 4 documents. (openai.py:196) [2025-02-27 07:25:34] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Binance-Architecture-Diagram_3748397105.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:34] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Binance-DNS-Record-in-Cloudflare_3743416389.html (crack_and_chunk.py:127) [2025-02-27 07:25:35] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityCompleted: Activity=Embeddings.embed, HowEnded=Success, Duration=61056.38 [ms] (activity.py:129) [2025-02-27 07:25:35] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:25:35] INFO azureml.rag.chunk_embedder_1 - Embedding took 61.65841221809387 seconds (embed.py:184) [2025-02-27 07:25:35] INFO azureml.rag.chunk_embedder_1 - Only data will be saved (embed.py:200) [2025-02-27 07:25:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:35] INFO azureml.rag.chunk_embedder_1 - waiting for chunk_batch: 1 (embed.py:159) [2025-02-27 07:25:35] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Binance-DNS-Record-in-Cloudflare_3743416389.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bitwarden-Setup-for-Secure-Login-Management-and-Data-Protection_3519741999.html (crack_and_chunk.py:127) [2025-02-27 07:25:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:37] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bitwarden-Setup-for-Secure-Login-Management-and-Data-Protection_3519741999.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:37] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bot-Management_3777298448.html (crack_and_chunk.py:127) [2025-02-27 07:25:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bot-Management_3777298448.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bring-your-own-Device-Guide_3773333516.html (crack_and_chunk.py:127) [2025-02-27 07:25:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bring-your-own-Device-Guide_3773333516.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:39] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Brute-force-Directories_3816062980.html (crack_and_chunk.py:127) [2025-02-27 07:25:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:41] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Brute-force-Directories_3816062980.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:41] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bruteforce-Web-Login_3816423431.html (crack_and_chunk.py:127) [2025-02-27 07:25:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:43] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bruteforce-Web-Login_3816423431.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:43] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bukit-Seri-Ceylon-Network-Information_2968846446.html (crack_and_chunk.py:127) [2025-02-27 07:25:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bukit-Seri-Ceylon-Network-Information_2968846446.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bukit-Seri-Ceylon-Network-Topology_3220144130.html (crack_and_chunk.py:127) [2025-02-27 07:25:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:46] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bukit-Seri-Ceylon-Network-Topology_3220144130.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Bukit-Seri-Ceylon-Office_3104800973.html (crack_and_chunk.py:127) [2025-02-27 07:25:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Bukit-Seri-Ceylon-Office_3104800973.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:49] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 200/200 processed_sources (logging.py:383) [2025-02-27 07:25:51] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 257/257 documents_total (logging.py:383) [2025-02-27 07:25:52] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:25:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Burp-Suite-Community-Edition---Free_3793682568.html (crack_and_chunk.py:127) [2025-02-27 07:25:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Burp-Suite-Community-Edition---Free_3793682568.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Certification-fdt.internal-installations-for-BYOD_3514531842.html (crack_and_chunk.py:127) [2025-02-27 07:25:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:58] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Certification-fdt.internal-installations-for-BYOD_3514531842.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:25:58] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Change-Management-Adaptation-Guide_3600646160.html (crack_and_chunk.py:127) [2025-02-27 07:25:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:25:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Change-Management-Adaptation-Guide_3600646160.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Change-Management-Policy_3565224047.html (crack_and_chunk.py:127) [2025-02-27 07:26:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Change-Management-Policy_3565224047.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/ChatGPT-Route-for-HK-Office_3252781120.html (crack_and_chunk.py:127) [2025-02-27 07:26:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/ChatGPT-Route-for-HK-Office_3252781120.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Chatbox-AI-Setup_3572498479.html (crack_and_chunk.py:127) [2025-02-27 07:26:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Chatbox-AI-Setup_3572498479.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:04] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cheapest-Prepaid-Data-Plans-for-Companies-in-Malaysia_3785981957.html (crack_and_chunk.py:127) [2025-02-27 07:26:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cheapest-Prepaid-Data-Plans-for-Companies-in-Malaysia_3785981957.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Check-Point---Email-Security_3875078178.html (crack_and_chunk.py:127) [2025-02-27 07:26:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Check-Point---Email-Security_3875078178.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:07] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Chime-V5_3638427652.html (crack_and_chunk.py:127) [2025-02-27 07:26:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Chime-V5_3638427652.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:07] INFO azureml.rag.crack_and_chunk - Processing file: TRM/ChimeV5-Architecture-Diagram_3638394888.html (crack_and_chunk.py:127) [2025-02-27 07:26:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:09] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/ChimeV5-Architecture-Diagram_3638394888.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:11] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 210/210 processed_sources (logging.py:383) [2025-02-27 07:26:12] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 267/267 documents_total (logging.py:383) [2025-02-27 07:26:13] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:26:13] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cisco-WebUI-SNMP-Configuration_3522068485.html (crack_and_chunk.py:127) [2025-02-27 07:26:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cisco-WebUI-SNMP-Configuration_3522068485.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Clear-Office365-Cache-Script_3859939338.html (crack_and_chunk.py:127) [2025-02-27 07:26:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Clear-Office365-Cache-Script_3859939338.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Connector_3777298435.html (crack_and_chunk.py:127) [2025-02-27 07:26:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Connector_3777298435.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Services-Lifecycle-Management-Procedure_3242229833.html (crack_and_chunk.py:127) [2025-02-27 07:26:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Services-Lifecycle-Management-Procedure_3242229833.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:19] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Storage-Security-Antivirus-S3-Alerts_3549397030.html (crack_and_chunk.py:127) [2025-02-27 07:26:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:21] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Storage-Security-Antivirus-S3-Alerts_3549397030.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:21] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Storage-Security-Antivirus-S3_3549429796.html (crack_and_chunk.py:127) [2025-02-27 07:26:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Storage-Security-Antivirus-S3_3549429796.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Storage-Security-DLP-Classification-Rule-Sets_3551232008.html (crack_and_chunk.py:127) [2025-02-27 07:26:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:24] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Storage-Security-DLP-Classification-Rule-Sets_3551232008.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:24] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Storage-Security-For-AWS-S3---Problem-Files_3529965604.html (crack_and_chunk.py:127) [2025-02-27 07:26:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Storage-Security-For-AWS-S3---Problem-Files_3529965604.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Storage-Security-Protected-by-another-Console-Issue_3572236291.html (crack_and_chunk.py:127) [2025-02-27 07:26:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Storage-Security-Protected-by-another-Console-Issue_3572236291.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Storage-Security-Release-Notes_3554148373.html (crack_and_chunk.py:127) [2025-02-27 07:26:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Storage-Security-Release-Notes_3554148373.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:28] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 220/220 processed_sources (logging.py:383) [2025-02-27 07:26:29] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 279/279 documents_total (logging.py:383) [2025-02-27 07:26:30] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:26:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloud-Storage-Security-S3-Teams-Channel-Alerts_3542122524.html (crack_and_chunk.py:127) [2025-02-27 07:26:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloud-Storage-Security-S3-Teams-Channel-Alerts_3542122524.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:31] INFO azureml.rag.crack_and_chunk - Processing file: TRM/CloudWatch-Log-Group-Export-to-S3-Bucket_3526426654.html (crack_and_chunk.py:127) [2025-02-27 07:26:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/CloudWatch-Log-Group-Export-to-S3-Bucket_3526426654.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:32] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloudflare-DDoS-Protection-How-to-Config_3516465226.html (crack_and_chunk.py:127) [2025-02-27 07:26:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:34] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloudflare-DDoS-Protection-How-to-Config_3516465226.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:34] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloudflare_3777200131.html (crack_and_chunk.py:127) [2025-02-27 07:26:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:35] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloudflare_3777200131.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cloudfront-TLS-and-HTTP-version-Upgrade-for-TimesTrust-Account_3787194375.html (crack_and_chunk.py:127) [2025-02-27 07:26:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:36] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cloudfront-TLS-and-HTTP-version-Upgrade-for-TimesTrust-Account_3787194375.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:36] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Colleen---Research_3612901408.html (crack_and_chunk.py:127) [2025-02-27 07:26:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Colleen---Research_3612901408.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Communication-Software_3282239491.html (crack_and_chunk.py:127) [2025-02-27 07:26:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Communication-Software_3282239491.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:39] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Company-Phones_3358916615.html (crack_and_chunk.py:127) [2025-02-27 07:26:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Company-Phones_3358916615.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Conclude---Slack-X-Teams_3808231426.html (crack_and_chunk.py:127) [2025-02-27 07:26:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:41] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Conclude---Slack-X-Teams_3808231426.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:41] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Confluence-Cloud-for-Microsoft-Teams-App-Meeting-Agenda_3728965635.html (crack_and_chunk.py:127) [2025-02-27 07:26:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Confluence-Cloud-for-Microsoft-Teams-App-Meeting-Agenda_3728965635.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:43] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 230/230 processed_sources (logging.py:383) [2025-02-27 07:26:45] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 290/290 documents_total (logging.py:383) [2025-02-27 07:26:46] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:26:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Control-of-Documented-Information-Procedure---ISO-27001_3288203283.html (crack_and_chunk.py:127) [2025-02-27 07:26:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Control-of-Documented-Information-Procedure---ISO-27001_3288203283.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Corporate-iPhone-set-up_3678470145.html (crack_and_chunk.py:127) [2025-02-27 07:26:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Corporate-iPhone-set-up_3678470145.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:49] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Create-an-AWS-account_3602415619.html (crack_and_chunk.py:127) [2025-02-27 07:26:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Create-an-AWS-account_3602415619.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Create-firstdigitallabs.com-Shared-Email_3384115207.html (crack_and_chunk.py:127) [2025-02-27 07:26:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:51] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Create-firstdigitallabs.com-Shared-Email_3384115207.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Customer-Data-Retention%2C-Extraction-and-Removal-Procedure_2908028986.html (crack_and_chunk.py:127) [2025-02-27 07:26:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Customer-Data-Retention%2C-Extraction-and-Removal-Procedure_2908028986.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Customize-channel-notifications-in-Microsoft-Teams_3587801089.html (crack_and_chunk.py:127) [2025-02-27 07:26:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Customize-channel-notifications-in-Microsoft-Teams_3587801089.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cyber-Incident-Response_3526033409.html (crack_and_chunk.py:127) [2025-02-27 07:26:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cyber-Incident-Response_3526033409.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cyber-Incident-Test-Cases_3533898208.html (crack_and_chunk.py:127) [2025-02-27 07:26:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cyber-Incident-Test-Cases_3533898208.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html (crack_and_chunk.py:127) [2025-02-27 07:26:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:26:57] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:26:57] INFO azureml.rag.embed - ==== Putting batch_id=2 with 100 chunks in queue (embed.py:320) [2025-02-27 07:27:02] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Embedded Documents - 200/300 documents_embedded (logging.py:383) [2025-02-27 07:27:02] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Total Documents - 200/300 documents_total (logging.py:383) [2025-02-27 07:27:02] INFO azureml.rag.chunk_embedder_0 - ==== embedding batch_id=2 with 100 chunks (embed.py:166) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF-Free-Rule-Group_3519217734.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AWS-WAF_3519348850.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Access-Management-for-FDT-Fortinet-Firewalls._3281256454.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Access-Management_2880634881.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Activate-Bitwarden-account_3161751583.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Add-FD121-Email-Account-in-Windows_3848994817.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Adding-FDT-Company-Holidays-Calendar-to-Your-Outlook_3298394485.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/All-EC2-IMDSv2-Upgrade-for-TimesTrust-Account_3786801229.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/All-Sub-Page-Missing-Header-Image_3790667798.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html1 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html2 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html3 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html4 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html5 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.crack_and_chunk - Processing file: TRM/DISM-Auto-Script_3668967457.html (crack_and_chunk.py:127) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html6 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html7 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html8 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html9 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html10 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/AllSecureVPN-Script-Deployment-to-Laptops_3874750479.html11 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Amazon-Q---Integration-with-Confluence_3797614646.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Application-Flow_3526623233.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Application-List_3483041818.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Architecture-of-the-External-Database-Backup-Pipeline_3210018832.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Architecture-of-the-External-Database-Backup-Pipeline_3210018832.html1 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Area-1-Email-Security-Enhances_3798007850.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Argo-Smart-Routing_3776839735.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Assign-Foxit-PDF-Editor-Perpetual-License-to-FDT-Staff_3407347791.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Atlassian-Confluence-Cloud-Behind-Firewall_3769466882.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Atlassian-Intelligence-features-in-Confluence_3797549060.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Atlassian_3574005810.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Azure-OpenAI-Service_3559063652.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Azure-OpenAI-vs-ChatGPT_3549298881.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Azure-OpenAI-vs-ChatGPT_3549298881.html1 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Azure-OpenAI-vs-ChatGPT_3549298881.html2 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Azure-OpenAI-vs-ChatGPT_3549298881.html3 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Azure-OpenAI_3572334655.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/BYOD-MacOS-Enrollment-Guide_3296788497.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/BYOD-MacOS-Unenroll-Device-Guide_3796107417.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/BYOD-Windows-Enrollment-Guide_3297181721.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Basic-WAF-Web-ACL-for-TimesTrust-Account_3787161632.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bastion-Server-Maintenance-for-TimesTrust_3786932280.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Best-Practices-for-Keeping-Your-Google-Account-Secure_3796860955.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Billing-Profile-Separate_3891429378.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Binance-AWS-Project_3681222658.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Binance-Architecture-Diagram_3748397105.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Binance-DNS-Record-in-Cloudflare_3743416389.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bitwarden-Setup-for-Secure-Login-Management-and-Data-Protection_3519741999.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bot-Management_3777298448.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bring-your-own-Device-Guide_3773333516.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Brute-force-Directories_3816062980.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bruteforce-Web-Login_3816423431.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bukit-Seri-Ceylon-Network-Information_2968846446.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bukit-Seri-Ceylon-Network-Information_2968846446.html1 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bukit-Seri-Ceylon-Network-Topology_3220144130.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Bukit-Seri-Ceylon-Office_3104800973.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Burp-Suite-Community-Edition---Free_3793682568.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Certification-fdt.internal-installations-for-BYOD_3514531842.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Change-Management-Adaptation-Guide_3600646160.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Change-Management-Policy_3565224047.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ChatGPT-Route-for-HK-Office_3252781120.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Chatbox-AI-Setup_3572498479.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cheapest-Prepaid-Data-Plans-for-Companies-in-Malaysia_3785981957.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Check-Point---Email-Security_3875078178.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Chime-V5_3638427652.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ChimeV5-Architecture-Diagram_3638394888.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cisco-WebUI-SNMP-Configuration_3522068485.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Clear-Office365-Cache-Script_3859939338.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Connector_3777298435.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Services-Lifecycle-Management-Procedure_3242229833.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-Antivirus-S3-Alerts_3549397030.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-Antivirus-S3_3549429796.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-DLP-Classification-Rule-Sets_3551232008.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-For-AWS-S3---Problem-Files_3529965604.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-For-AWS-S3---Problem-Files_3529965604.html1 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-For-AWS-S3---Problem-Files_3529965604.html2 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-Protected-by-another-Console-Issue_3572236291.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-Release-Notes_3554148373.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloud-Storage-Security-S3-Teams-Channel-Alerts_3542122524.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/CloudWatch-Log-Group-Export-to-S3-Bucket_3526426654.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloudflare-DDoS-Protection-How-to-Config_3516465226.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloudflare_3777200131.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cloudfront-TLS-and-HTTP-version-Upgrade-for-TimesTrust-Account_3787194375.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Colleen---Research_3612901408.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Communication-Software_3282239491.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Company-Phones_3358916615.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Conclude---Slack-X-Teams_3808231426.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Conclude---Slack-X-Teams_3808231426.html1 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Confluence-Cloud-for-Microsoft-Teams-App-Meeting-Agenda_3728965635.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Control-of-Documented-Information-Procedure---ISO-27001_3288203283.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Corporate-iPhone-set-up_3678470145.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Create-an-AWS-account_3602415619.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Create-firstdigitallabs.com-Shared-Email_3384115207.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Customer-Data-Retention%2C-Extraction-and-Removal-Procedure_2908028986.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Customize-channel-notifications-in-Microsoft-Teams_3587801089.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cyber-Incident-Response_3526033409.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cyber-Incident-Test-Cases_3533898208.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cyber-Incident-Test-Cases_3533898208.html1 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html0 (__init__.py:1300) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings - Documents to embed: 100 Documents reused: 0 (__init__.py:1337) [2025-02-27 07:27:02] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityStarted, Embeddings.embed (activity.py:108) [2025-02-27 07:27:02] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:27:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:03] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:27:03] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:27:04] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:27:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/DISM-Auto-Script_3668967457.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:04] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:27:04] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:27:05] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 4 documents. (openai.py:196) [2025-02-27 07:27:05] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 240/240 processed_sources (logging.py:383) [2025-02-27 07:27:07] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 311/311 documents_total (logging.py:383) [2025-02-27 07:27:08] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:27:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/DLP-Classification-Result-Action_3580854277.html (crack_and_chunk.py:127) [2025-02-27 07:27:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/DLP-Classification-Result-Action_3580854277.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Data-Privacy-Policy_3583115265.html (crack_and_chunk.py:127) [2025-02-27 07:27:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Data-Privacy-Policy_3583115265.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Data-Protection-Policy_3793256449.html (crack_and_chunk.py:127) [2025-02-27 07:27:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Data-Protection-Policy_3793256449.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Delete-Bitwarden-account-if-not-linked-with-the-organization_3509354577.html (crack_and_chunk.py:127) [2025-02-27 07:27:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:12] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Delete-Bitwarden-account-if-not-linked-with-the-organization_3509354577.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Demo-Launch-Global-Secure-Access-After-Configure_3429007557.html (crack_and_chunk.py:127) [2025-02-27 07:27:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:13] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Demo-Launch-Global-Secure-Access-After-Configure_3429007557.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:13] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Deploy-FortiClient-AllSecureVPN-Configs-Script_3860004890.html (crack_and_chunk.py:127) [2025-02-27 07:27:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:14] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Deploy-FortiClient-AllSecureVPN-Configs-Script_3860004890.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:14] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Determ---Social-media-monitoring_3818258435.html (crack_and_chunk.py:127) [2025-02-27 07:27:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Determ---Social-media-monitoring_3818258435.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/DevSecOps-Implementation_3728965653.html (crack_and_chunk.py:127) [2025-02-27 07:27:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/DevSecOps-Implementation_3728965653.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Disaster-Recovery-Drill-Reports_3209986527.html (crack_and_chunk.py:127) [2025-02-27 07:27:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Disaster-Recovery-Drill-Reports_3209986527.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:19] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Disaster-Recovery-Runbook_3165421605.html (crack_and_chunk.py:127) [2025-02-27 07:27:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Disaster-Recovery-Runbook_3165421605.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:22] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 250/250 processed_sources (logging.py:383) [2025-02-27 07:27:24] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 326/326 documents_total (logging.py:383) [2025-02-27 07:27:25] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:27:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Disaster-Recovery-Test-Cases_3194257460.html (crack_and_chunk.py:127) [2025-02-27 07:27:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Disaster-Recovery-Test-Cases_3194257460.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/DoIT---AWS-Partner_3602382850.html (crack_and_chunk.py:127) [2025-02-27 07:27:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:26] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/DoIT---AWS-Partner_3602382850.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/DoiT-Portal-Installation-EKS-Cluster-Advanced-Insight-into-your-Kubernetes-Spend_3448995861.html (crack_and_chunk.py:127) [2025-02-27 07:27:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/DoiT-Portal-Installation-EKS-Cluster-Advanced-Insight-into-your-Kubernetes-Spend_3448995861.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Draft---Technology-Risk..._3797090310.html (crack_and_chunk.py:127) [2025-02-27 07:27:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Draft---Technology-Risk..._3797090310.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:31] INFO azureml.rag.crack_and_chunk - Processing file: TRM/EC2-Volume-Upgrade-Type-For-TimesTrust-Account_3787358209.html (crack_and_chunk.py:127) [2025-02-27 07:27:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/EC2-Volume-Upgrade-Type-For-TimesTrust-Account_3787358209.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:32] INFO azureml.rag.crack_and_chunk - Processing file: TRM/ECR-Setup-Lifecyle-For-TimesTrust-Account_3787161610.html (crack_and_chunk.py:127) [2025-02-27 07:27:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:35] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/ECR-Setup-Lifecyle-For-TimesTrust-Account_3787161610.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/EIP-EC2-Public-Instance-for-TimesTrust-Account_3787390981.html (crack_and_chunk.py:127) [2025-02-27 07:27:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:36] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/EIP-EC2-Public-Instance-for-TimesTrust-Account_3787390981.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:36] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Email-Distribution-Lists-Overview---allstaff.notify@1stdigital.com_3459285009.html (crack_and_chunk.py:127) [2025-02-27 07:27:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Email-Distribution-Lists-Overview---allstaff.notify@1stdigital.com_3459285009.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Email-Security-Vendor-Comparison_3805708476.html (crack_and_chunk.py:127) [2025-02-27 07:27:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Email-Security-Vendor-Comparison_3805708476.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Email-Template-asking-for-using-personal-or-company-assets_3809902607.html (crack_and_chunk.py:127) [2025-02-27 07:27:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Email-Template-asking-for-using-personal-or-company-assets_3809902607.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:44] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 260/260 processed_sources (logging.py:383) [2025-02-27 07:27:46] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 342/342 documents_total (logging.py:383) [2025-02-27 07:27:47] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:27:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Email-Template-to-Remote-Worker-for-Company-Assets_3791880246.html (crack_and_chunk.py:127) [2025-02-27 07:27:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Email-Template-to-Remote-Worker-for-Company-Assets_3791880246.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:49] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Enable-MFA-for-TimesTrust-AWS-IAM-Account_3786473474.html (crack_and_chunk.py:127) [2025-02-27 07:27:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:49] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Enable-MFA-for-TimesTrust-AWS-IAM-Account_3786473474.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Enable-Passwordless-Method---Phone-Sign-in_3257696257.html (crack_and_chunk.py:127) [2025-02-27 07:27:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Enable-Passwordless-Method---Phone-Sign-in_3257696257.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Encrypted-Connections-for-Production-Environment-Access_3282272273.html (crack_and_chunk.py:127) [2025-02-27 07:27:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Encrypted-Connections-for-Production-Environment-Access_3282272273.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Essential-Tools-in-Kali-Linux_3790209063.html (crack_and_chunk.py:127) [2025-02-27 07:27:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Essential-Tools-in-Kali-Linux_3790209063.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FD121-Portal-Access-Guide_3846963229.html (crack_and_chunk.py:127) [2025-02-27 07:27:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:58] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FD121-Portal-Access-Guide_3846963229.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:27:58] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FDT-Application-API-Monitoring_3501424688.html (crack_and_chunk.py:127) [2025-02-27 07:27:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:27:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FDT-Application-API-Monitoring_3501424688.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FDT-Audio-Equipment-Setup-for-Townhall-in-KL_3718676488.html (crack_and_chunk.py:127) [2025-02-27 07:28:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:02] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FDT-Audio-Equipment-Setup-for-Townhall-in-KL_3718676488.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:02] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FDT-KL-Menara-Prestige-Office-Floor-Plan_3291545623.html (crack_and_chunk.py:127) [2025-02-27 07:28:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:03] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityCompleted: Activity=Embeddings.embed, HowEnded=Success, Duration=60746.68 [ms] (activity.py:129) [2025-02-27 07:28:03] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:28:03] INFO azureml.rag.chunk_embedder_0 - Embedding took 60.81263470649719 seconds (embed.py:184) [2025-02-27 07:28:03] INFO azureml.rag.chunk_embedder_0 - Only data will be saved (embed.py:200) [2025-02-27 07:28:03] INFO azureml.rag.chunk_embedder_0 - waiting for chunk_batch: 0 (embed.py:159) [2025-02-27 07:28:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FDT-KL-Menara-Prestige-Office-Floor-Plan_3291545623.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:04] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FDT-Office-Networks_3281518618.html (crack_and_chunk.py:127) [2025-02-27 07:28:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FDT-Office-Networks_3281518618.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:07] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 270/270 processed_sources (logging.py:383) [2025-02-27 07:28:09] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 352/352 documents_total (logging.py:383) [2025-02-27 07:28:11] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:28:11] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Facility-and-Equipment-Guidelines-for-IT-Operations_2970288141.html (crack_and_chunk.py:127) [2025-02-27 07:28:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:12] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Facility-and-Equipment-Guidelines-for-IT-Operations_2970288141.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Firewall-Rule-Review-Procedure_3525345316.html (crack_and_chunk.py:127) [2025-02-27 07:28:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:14] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Firewall-Rule-Review-Procedure_3525345316.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:14] INFO azureml.rag.crack_and_chunk - Processing file: TRM/First-Digital-Labs-Procedures_3847159813.html (crack_and_chunk.py:127) [2025-02-27 07:28:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:14] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/First-Digital-Labs-Procedures_3847159813.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:14] INFO azureml.rag.crack_and_chunk - Processing file: TRM/First-Digital-Trust-SIP_3256320030.html (crack_and_chunk.py:127) [2025-02-27 07:28:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/First-Digital-Trust-SIP_3256320030.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/First-Digital-Trust-SharePoint-for-Microsoft-Intune_3717595138.html (crack_and_chunk.py:127) [2025-02-27 07:28:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:16] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/First-Digital-Trust-SharePoint-for-Microsoft-Intune_3717595138.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/First-Digital-Trust-and-Suitable-Cybersecurity-Awareness-Training-Platforms_3459678254.html (crack_and_chunk.py:127) [2025-02-27 07:28:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:16] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/First-Digital-Trust-and-Suitable-Cybersecurity-Awareness-Training-Platforms_3459678254.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Fix-Outlook-Classic-Inbox-Sync-Issue_3690201101.html (crack_and_chunk.py:127) [2025-02-27 07:28:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Fix-Outlook-Classic-Inbox-Sync-Issue_3690201101.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:19] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html (crack_and_chunk.py:127) [2025-02-27 07:28:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FortiGate-120G-Quotation-Comparison_3672408066.html (crack_and_chunk.py:127) [2025-02-27 07:28:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:21] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FortiGate-120G-Quotation-Comparison_3672408066.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:21] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FortiGate-Active-Active-vs-Active-Passive-mode_3712450588.html (crack_and_chunk.py:127) [2025-02-27 07:28:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FortiGate-Active-Active-vs-Active-Passive-mode_3712450588.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:24] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 280/280 processed_sources (logging.py:383) [2025-02-27 07:28:25] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 391/391 documents_total (logging.py:383) [2025-02-27 07:28:27] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:28:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/FortiGate-IPsec-VPN-with-Microsoft-Entra-ID-SAML-Authentication_3833495553.html (crack_and_chunk.py:127) [2025-02-27 07:28:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:29] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/FortiGate-IPsec-VPN-with-Microsoft-Entra-ID-SAML-Authentication_3833495553.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:29] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Fortinet-Firewall-Let%27s-Encrypt-Certificate-Installation-Guide_3341778949.html (crack_and_chunk.py:127) [2025-02-27 07:28:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Fortinet-Firewall-Let%27s-Encrypt-Certificate-Installation-Guide_3341778949.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:31] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Fortinet-Support-Information_3467116548.html (crack_and_chunk.py:127) [2025-02-27 07:28:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Fortinet-Support-Information_3467116548.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:32] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Foxit-PDF-Editor-Activation-Guide-with-SSO-Login_3415801865.html (crack_and_chunk.py:127) [2025-02-27 07:28:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Foxit-PDF-Editor-Activation-Guide-with-SSO-Login_3415801865.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:32] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Foxit-PDF-Editor-vs-Adobe-Acrobat-Pro_3407052895.html (crack_and_chunk.py:127) [2025-02-27 07:28:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:35] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Foxit-PDF-Editor-vs-Adobe-Acrobat-Pro_3407052895.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Foxit-PDF-Editor_3407052842.html (crack_and_chunk.py:127) [2025-02-27 07:28:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:37] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Foxit-PDF-Editor_3407052842.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:37] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Foxit-PhantomPDF---Alternative-to-Adobe-Acrobat-Pro_3365797908.html (crack_and_chunk.py:127) [2025-02-27 07:28:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Foxit-PhantomPDF---Alternative-to-Adobe-Acrobat-Pro_3365797908.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Global-Secure-Access_3429040167.html (crack_and_chunk.py:127) [2025-02-27 07:28:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Global-Secure-Access_3429040167.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Google_3797286916.html (crack_and_chunk.py:127) [2025-02-27 07:28:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:41] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Google_3797286916.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:41] INFO azureml.rag.embed - ==== Putting batch_id=3 with 100 chunks in queue (embed.py:320) [2025-02-27 07:28:45] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Embedded Documents - 300/400 documents_embedded (logging.py:383) [2025-02-27 07:28:46] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Total Documents - 300/400 documents_total (logging.py:383) [2025-02-27 07:28:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Grafana-Dashboard_3491430402.html (crack_and_chunk.py:127) [2025-02-27 07:28:46] INFO azureml.rag.chunk_embedder_1 - ==== embedding batch_id=3 with 100 chunks (embed.py:166) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html2 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html3 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html4 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html5 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html6 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html7 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html8 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html9 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Cybersecurity-and-Information-Security-Policy_3554115688.html10 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/DISM-Auto-Script_3668967457.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/DLP-Classification-Result-Action_3580854277.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Data-Privacy-Policy_3583115265.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Data-Privacy-Policy_3583115265.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Data-Privacy-Policy_3583115265.html2 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Data-Privacy-Policy_3583115265.html3 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Data-Protection-Policy_3793256449.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Data-Protection-Policy_3793256449.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Delete-Bitwarden-account-if-not-linked-with-the-organization_3509354577.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Demo-Launch-Global-Secure-Access-After-Configure_3429007557.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Deploy-FortiClient-AllSecureVPN-Configs-Script_3860004890.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Determ---Social-media-monitoring_3818258435.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/DevSecOps-Implementation_3728965653.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Disaster-Recovery-Drill-Reports_3209986527.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Disaster-Recovery-Runbook_3165421605.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Disaster-Recovery-Runbook_3165421605.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Disaster-Recovery-Test-Cases_3194257460.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Disaster-Recovery-Test-Cases_3194257460.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/DoIT---AWS-Partner_3602382850.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/DoiT-Portal-Installation-EKS-Cluster-Advanced-Insight-into-your-Kubernetes-Spend_3448995861.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Draft---Technology-Risk..._3797090310.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Draft---Technology-Risk..._3797090310.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Draft---Technology-Risk..._3797090310.html2 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Draft---Technology-Risk..._3797090310.html3 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Draft---Technology-Risk..._3797090310.html4 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Draft---Technology-Risk..._3797090310.html5 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/EC2-Volume-Upgrade-Type-For-TimesTrust-Account_3787358209.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ECR-Setup-Lifecyle-For-TimesTrust-Account_3787161610.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/EIP-EC2-Public-Instance-for-TimesTrust-Account_3787390981.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Email-Distribution-Lists-Overview---allstaff.notify@1stdigital.com_3459285009.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Email-Security-Vendor-Comparison_3805708476.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Email-Template-asking-for-using-personal-or-company-assets_3809902607.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Email-Template-to-Remote-Worker-for-Company-Assets_3791880246.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Enable-MFA-for-TimesTrust-AWS-IAM-Account_3786473474.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Enable-Passwordless-Method---Phone-Sign-in_3257696257.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Encrypted-Connections-for-Production-Environment-Access_3282272273.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Essential-Tools-in-Kali-Linux_3790209063.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FD121-Portal-Access-Guide_3846963229.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FDT-Application-API-Monitoring_3501424688.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FDT-Audio-Equipment-Setup-for-Townhall-in-KL_3718676488.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FDT-KL-Menara-Prestige-Office-Floor-Plan_3291545623.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FDT-Office-Networks_3281518618.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Facility-and-Equipment-Guidelines-for-IT-Operations_2970288141.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Firewall-Rule-Review-Procedure_3525345316.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/First-Digital-Labs-Procedures_3847159813.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/First-Digital-Trust-SIP_3256320030.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/First-Digital-Trust-SharePoint-for-Microsoft-Intune_3717595138.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/First-Digital-Trust-SharePoint-for-Microsoft-Intune_3717595138.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/First-Digital-Trust-and-Suitable-Cybersecurity-Awareness-Training-Platforms_3459678254.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/First-Digital-Trust-and-Suitable-Cybersecurity-Awareness-Training-Platforms_3459678254.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Fix-Outlook-Classic-Inbox-Sync-Issue_3690201101.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html1 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html2 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html3 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html4 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html5 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html6 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html7 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html8 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html9 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html10 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html11 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html12 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html13 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html14 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html15 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html16 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html17 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html18 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html19 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html20 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html21 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html22 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html23 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html24 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html25 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html26 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiClient-VPN-Configuration-for-Deployment_3341942796.html27 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiGate-120G-Quotation-Comparison_3672408066.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiGate-Active-Active-vs-Active-Passive-mode_3712450588.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/FortiGate-IPsec-VPN-with-Microsoft-Entra-ID-SAML-Authentication_3833495553.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Fortinet-Firewall-Let%27s-Encrypt-Certificate-Installation-Guide_3341778949.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Fortinet-Support-Information_3467116548.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Foxit-PDF-Editor-Activation-Guide-with-SSO-Login_3415801865.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Foxit-PDF-Editor-vs-Adobe-Acrobat-Pro_3407052895.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Foxit-PDF-Editor_3407052842.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Foxit-PhantomPDF---Alternative-to-Adobe-Acrobat-Pro_3365797908.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Global-Secure-Access_3429040167.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Google_3797286916.html0 (__init__.py:1300) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings - Documents to embed: 100 Documents reused: 0 (__init__.py:1337) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityStarted, Embeddings.embed (activity.py:108) [2025-02-27 07:28:46] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:47] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:28:47] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:28:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Grafana-Dashboard_3491430402.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:48] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:28:48] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:28:49] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 290/290 processed_sources (logging.py:383) [2025-02-27 07:28:51] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 401/401 documents_total (logging.py:383) [2025-02-27 07:28:52] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:28:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Grafana-Report---2024_3693477919.html (crack_and_chunk.py:127) [2025-02-27 07:28:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Grafana-Report---2024_3693477919.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Grafana-Report---2025_3810295862.html (crack_and_chunk.py:127) [2025-02-27 07:28:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:55] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Grafana-Report---2025_3810295862.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Grafana-Report_3693314067.html (crack_and_chunk.py:127) [2025-02-27 07:28:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Grafana-Report_3693314067.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/GuardDuty-Findings-Join-Binance-AWS-Account_3761504260.html (crack_and_chunk.py:127) [2025-02-27 07:28:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:28:59] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/GuardDuty-Findings-Join-Binance-AWS-Account_3761504260.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:28:59] INFO azureml.rag.crack_and_chunk - Processing file: TRM/HK-FortiGate-Vendor-Firewall-Comparison_3692920843.html (crack_and_chunk.py:127) [2025-02-27 07:29:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:01] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/HK-FortiGate-Vendor-Firewall-Comparison_3692920843.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:01] INFO azureml.rag.crack_and_chunk - Processing file: TRM/HK-Guest-Wi-Fi_3752722435.html (crack_and_chunk.py:127) [2025-02-27 07:29:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/HK-Guest-Wi-Fi_3752722435.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/HK-Office-Network-Information_3260645500.html (crack_and_chunk.py:127) [2025-02-27 07:29:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/HK-Office-Network-Information_3260645500.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/HKBN-Internet-Set-up-for-FortiGate-Firewall_3755704322.html (crack_and_chunk.py:127) [2025-02-27 07:29:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/HKBN-Internet-Set-up-for-FortiGate-Firewall_3755704322.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:04] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hardware-purchase-from-vendor_2968846508.html (crack_and_chunk.py:127) [2025-02-27 07:29:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hardware-purchase-from-vendor_2968846508.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hardware_3105357917.html (crack_and_chunk.py:127) [2025-02-27 07:29:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hardware_3105357917.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:09] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 300/300 processed_sources (logging.py:383) [2025-02-27 07:29:10] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 413/413 documents_total (logging.py:383) [2025-02-27 07:29:12] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:29:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hong-Kong-Firewall-Log_3578757156.html (crack_and_chunk.py:127) [2025-02-27 07:29:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hong-Kong-Firewall-Log_3578757156.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hong-Kong-Firewall-Memory-Issue_3569680563.html (crack_and_chunk.py:127) [2025-02-27 07:29:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hong-Kong-Firewall-Memory-Issue_3569680563.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hong-Kong-Firewall-Plan-Upgrade---Single-VDOM-Managing-Multiple-VLANs_3492806693.html (crack_and_chunk.py:127) [2025-02-27 07:29:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hong-Kong-Firewall-Plan-Upgrade---Single-VDOM-Managing-Multiple-VLANs_3492806693.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hong-Kong-Firewall-Upgrade-Proposal_3569680610.html (crack_and_chunk.py:127) [2025-02-27 07:29:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hong-Kong-Firewall-Upgrade-Proposal_3569680610.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hong-Kong-Office-Floor-Plan_3583344643.html (crack_and_chunk.py:127) [2025-02-27 07:29:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hong-Kong-Office-Floor-Plan_3583344643.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hong-Kong-Office_3260645470.html (crack_and_chunk.py:127) [2025-02-27 07:29:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:24] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hong-Kong-Office_3260645470.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:24] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Hotlink-Protection_3777200142.html (crack_and_chunk.py:127) [2025-02-27 07:29:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Hotlink-Protection_3777200142.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-To-Generate-HTML-Report-in-OWASP-ZAP_3813769264.html (crack_and_chunk.py:127) [2025-02-27 07:29:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-To-Generate-HTML-Report-in-OWASP-ZAP_3813769264.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-To-Generate-Individual-HTML-Report-in-OWASP-ZAP_3814621195.html (crack_and_chunk.py:127) [2025-02-27 07:29:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-To-Generate-Individual-HTML-Report-in-OWASP-ZAP_3814621195.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-To-Perform-An-Automated-Scan_3813572637.html (crack_and_chunk.py:127) [2025-02-27 07:29:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:33] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-To-Perform-An-Automated-Scan_3813572637.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:34] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 310/310 processed_sources (logging.py:383) [2025-02-27 07:29:34] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 425/425 documents_total (logging.py:383) [2025-02-27 07:29:36] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:29:36] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-To-Perform-An-Manual-Scanning_3813539879.html (crack_and_chunk.py:127) [2025-02-27 07:29:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:37] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-To-Perform-An-Manual-Scanning_3813539879.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:37] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-To-Scanning-Amazon-EC2-Instances-With-Amazon-Inspector_3457155076.html (crack_and_chunk.py:127) [2025-02-27 07:29:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-To-Scanning-Amazon-EC2-Instances-With-Amazon-Inspector_3457155076.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:39] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-To-Upgrade-WordPress-Version_3788505127.html (crack_and_chunk.py:127) [2025-02-27 07:29:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-To-Upgrade-WordPress-Version_3788505127.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-can-I-move-or-migrate-a-PRTG-installation-to-a-different-system-or-server_3740401754.html (crack_and_chunk.py:127) [2025-02-27 07:29:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-can-I-move-or-migrate-a-PRTG-installation-to-a-different-system-or-server_3740401754.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-Activate-an-eSIM-on-Company-iPhone_3753607177.html (crack_and_chunk.py:127) [2025-02-27 07:29:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:45] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-Activate-an-eSIM-on-Company-iPhone_3753607177.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:45] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-Add-Apps-for-iOS-via-Microsoft-Intune_3358359557.html (crack_and_chunk.py:127) [2025-02-27 07:29:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:46] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-Add-Apps-for-iOS-via-Microsoft-Intune_3358359557.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-Add-FD121-Email-Account-in-Windows_3819175946.html (crack_and_chunk.py:127) [2025-02-27 07:29:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-Add-FD121-Email-Account-in-Windows_3819175946.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-Create-a-Google-Account-Using-Your-Existing-1stdigital-Email_3798007910.html (crack_and_chunk.py:127) [2025-02-27 07:29:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:48] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:29:48] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 4 documents. (openai.py:196) [2025-02-27 07:29:48] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityCompleted: Activity=Embeddings.embed, HowEnded=Success, Duration=62424.04 [ms] (activity.py:129) [2025-02-27 07:29:48] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:29:48] INFO azureml.rag.chunk_embedder_1 - Embedding took 62.50489687919617 seconds (embed.py:184) [2025-02-27 07:29:48] INFO azureml.rag.chunk_embedder_1 - Only data will be saved (embed.py:200) [2025-02-27 07:29:48] INFO azureml.rag.chunk_embedder_1 - waiting for chunk_batch: 1 (embed.py:159) [2025-02-27 07:29:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-Create-a-Google-Account-Using-Your-Existing-1stdigital-Email_3798007910.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-Make-Teams-App-Available-to-Everyone-in-Organization-using-Microsoft-Teams-Admin-Center_3638624257.html (crack_and_chunk.py:127) [2025-02-27 07:29:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-Make-Teams-App-Available-to-Everyone-in-Organization-using-Microsoft-Teams-Admin-Center_3638624257.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-Setup-Oauth-2.0-Configuration-for-Confluence_3769335810.html (crack_and_chunk.py:127) [2025-02-27 07:29:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-Setup-Oauth-2.0-Configuration-for-Confluence_3769335810.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:29:55] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 320/320 processed_sources (logging.py:383) [2025-02-27 07:29:57] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 435/435 documents_total (logging.py:383) [2025-02-27 07:29:57] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:29:57] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-dial-an-outside-number_3339714565.html (crack_and_chunk.py:127) [2025-02-27 07:29:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:29:57] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-dial-an-outside-number_3339714565.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/How-to-set-up-OneDrive-API_3777298477.html (crack_and_chunk.py:127) [2025-02-27 07:30:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:01] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/How-to-set-up-OneDrive-API_3777298477.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:01] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IAM-User-Access-Keys-To-Rotate_3180003341.html (crack_and_chunk.py:127) [2025-02-27 07:30:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IAM-User-Access-Keys-To-Rotate_3180003341.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IPSec-Dial-up-Method-with-Firewall-Certificate-Authentication_3833790466.html (crack_and_chunk.py:127) [2025-02-27 07:30:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IPSec-Dial-up-Method-with-Firewall-Certificate-Authentication_3833790466.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IPsec-VPN---AllSecureVPN_3833102338.html (crack_and_chunk.py:127) [2025-02-27 07:30:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:06] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IPsec-VPN---AllSecureVPN_3833102338.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/ISMS-Board-of-Directors-Charter_3315335176.html (crack_and_chunk.py:127) [2025-02-27 07:30:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/ISMS-Board-of-Directors-Charter_3315335176.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:07] INFO azureml.rag.crack_and_chunk - Processing file: TRM/ISMS-Scope_3244032039.html (crack_and_chunk.py:127) [2025-02-27 07:30:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:08] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/ISMS-Scope_3244032039.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IT-Admin-Procedures_2977628161.html (crack_and_chunk.py:127) [2025-02-27 07:30:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:09] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IT-Admin-Procedures_2977628161.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:09] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IT-Disaster-Recovery-Plan_2909831169.html (crack_and_chunk.py:127) [2025-02-27 07:30:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IT-Disaster-Recovery-Plan_2909831169.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IT-Policies_2880733197.html (crack_and_chunk.py:127) [2025-02-27 07:30:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:11] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IT-Policies_2880733197.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:13] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 330/330 processed_sources (logging.py:383) [2025-02-27 07:30:14] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 447/447 documents_total (logging.py:383) [2025-02-27 07:30:15] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:30:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IT-Resilience_2880864264.html (crack_and_chunk.py:127) [2025-02-27 07:30:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IT-Resilience_2880864264.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IT-Risk-Register---FD121_3500146689.html (crack_and_chunk.py:127) [2025-02-27 07:30:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IT-Risk-Register---FD121_3500146689.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IT-Risk-Register---FDT_3425992930.html (crack_and_chunk.py:127) [2025-02-27 07:30:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IT-Risk-Register---FDT_3425992930.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/ITSM---JIRA-Service-Management-administration_2972778497.html (crack_and_chunk.py:127) [2025-02-27 07:30:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/ITSM---JIRA-Service-Management-administration_2972778497.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:22] INFO azureml.rag.crack_and_chunk - Processing file: TRM/ITSM-User-Guide_2915434534.html (crack_and_chunk.py:127) [2025-02-27 07:30:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/ITSM-User-Guide_2915434534.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Identity-Center-Azure-AD-SSO-for-TimesTrust-Account_3786637313.html (crack_and_chunk.py:127) [2025-02-27 07:30:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:24] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Identity-Center-Azure-AD-SSO-for-TimesTrust-Account_3786637313.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Implement-TimesTrust-AWS-Account_3786473581.html (crack_and_chunk.py:127) [2025-02-27 07:30:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Implement-TimesTrust-AWS-Account_3786473581.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Implementation-Steps_3516006405.html (crack_and_chunk.py:127) [2025-02-27 07:30:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Implementation-Steps_3516006405.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Incident-Management_3600220166.html (crack_and_chunk.py:127) [2025-02-27 07:30:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Incident-Management_3600220166.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Incident-Response-Procedures_3525640247.html (crack_and_chunk.py:127) [2025-02-27 07:30:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Incident-Response-Procedures_3525640247.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:33] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 340/340 processed_sources (logging.py:383) [2025-02-27 07:30:34] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 457/457 documents_total (logging.py:383) [2025-02-27 07:30:35] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:30:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Information-Asset-Management_3558735939.html (crack_and_chunk.py:127) [2025-02-27 07:30:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:37] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Information-Asset-Management_3558735939.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:37] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Information-Security-KRIs_3533602817.html (crack_and_chunk.py:127) [2025-02-27 07:30:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:37] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Information-Security-KRIs_3533602817.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Information-Security-Program_3342860289.html (crack_and_chunk.py:127) [2025-02-27 07:30:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Information-Security-Program_3342860289.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Information-Security_2880864271.html (crack_and_chunk.py:127) [2025-02-27 07:30:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:41] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Information-Security_2880864271.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:41] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Installing-ModHeader-in-Your-Browser_3526000663.html (crack_and_chunk.py:127) [2025-02-27 07:30:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Installing-ModHeader-in-Your-Browser_3526000663.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Instructions-to-Create-a-Passkey-for-DocuSign-Users_3821273149.html (crack_and_chunk.py:127) [2025-02-27 07:30:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:42] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Instructions-to-Create-a-Passkey-for-DocuSign-Users_3821273149.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Integration-of-AWS-Single-Sign-on-with-Azure-Active-Directory_3532587016.html (crack_and_chunk.py:127) [2025-02-27 07:30:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:44] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Integration-of-AWS-Single-Sign-on-with-Azure-Active-Directory_3532587016.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/IoT-Devices-Policy_3538976884.html (crack_and_chunk.py:127) [2025-02-27 07:30:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:45] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/IoT-Devices-Policy_3538976884.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:45] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Ironscales---Email-Security_3821043734.html (crack_and_chunk.py:127) [2025-02-27 07:30:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Ironscales---Email-Security_3821043734.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/January-9th-2025---25.1.102.1373_3834544136.html (crack_and_chunk.py:127) [2025-02-27 07:30:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:48] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/January-9th-2025---25.1.102.1373_3834544136.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:49] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 350/350 processed_sources (logging.py:383) [2025-02-27 07:30:50] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 474/474 documents_total (logging.py:383) [2025-02-27 07:30:52] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:30:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Jira-How-To-Remove-Label_3648258050.html (crack_and_chunk.py:127) [2025-02-27 07:30:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:53] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Jira-How-To-Remove-Label_3648258050.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:53] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Jira-Notification-Feed-Into-Microsoft-Teams-Channel_3620634626.html (crack_and_chunk.py:127) [2025-02-27 07:30:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:55] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Jira-Notification-Feed-Into-Microsoft-Teams-Channel_3620634626.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Jira-Reports-Tutorials_3646849036.html (crack_and_chunk.py:127) [2025-02-27 07:30:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Jira-Reports-Tutorials_3646849036.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Join-DoIT-Organization-for-TimesTrust-Account_3786932226.html (crack_and_chunk.py:127) [2025-02-27 07:30:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:59] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Join-DoIT-Organization-for-TimesTrust-Account_3786932226.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:30:59] INFO azureml.rag.crack_and_chunk - Processing file: TRM/KL-Guest-Wi-Fi_3590717447.html (crack_and_chunk.py:127) [2025-02-27 07:30:59] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:30:59] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:00] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/KL-Guest-Wi-Fi_3590717447.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:00] INFO azureml.rag.crack_and_chunk - Processing file: TRM/KL-Meeting-Room-Video-Conference-Set-up_3582427186.html (crack_and_chunk.py:127) [2025-02-27 07:31:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:01] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/KL-Meeting-Room-Video-Conference-Set-up_3582427186.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:01] INFO azureml.rag.crack_and_chunk - Processing file: TRM/KL-Menara-Prestige-Data-Ports_3580788740.html (crack_and_chunk.py:127) [2025-02-27 07:31:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:02] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/KL-Menara-Prestige-Data-Ports_3580788740.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/KL-Menara-Prestige-Meeting-Room-VC-Diagram_3413409810.html (crack_and_chunk.py:127) [2025-02-27 07:31:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/KL-Menara-Prestige-Meeting-Room-VC-Diagram_3413409810.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:04] INFO azureml.rag.crack_and_chunk - Processing file: TRM/KL-Menara-Prestige-Network-Device-Information_3217522719.html (crack_and_chunk.py:127) [2025-02-27 07:31:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/KL-Menara-Prestige-Network-Device-Information_3217522719.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/KRI_3693346853.html (crack_and_chunk.py:127) [2025-02-27 07:31:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/KRI_3693346853.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:08] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 360/360 processed_sources (logging.py:383) [2025-02-27 07:31:09] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 485/485 documents_total (logging.py:383) [2025-02-27 07:31:10] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:31:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Kali-Linux_3790045231.html (crack_and_chunk.py:127) [2025-02-27 07:31:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:12] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Kali-Linux_3790045231.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Kelvin---Research_3501391933.html (crack_and_chunk.py:127) [2025-02-27 07:31:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:13] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Kelvin---Research_3501391933.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:13] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Key-Rotation-on-AWS_3179773958.html (crack_and_chunk.py:127) [2025-02-27 07:31:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Key-Rotation-on-AWS_3179773958.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Kibana-vs.-Grafana---A-Scenario-Based-Decision-Guide_3500179460.html (crack_and_chunk.py:127) [2025-02-27 07:31:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Kibana-vs.-Grafana---A-Scenario-Based-Decision-Guide_3500179460.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Knowledge-Base-Documentation_3442147346.html (crack_and_chunk.py:127) [2025-02-27 07:31:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Knowledge-Base-Documentation_3442147346.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Laptop-Buyout-Request---FDT-Staff_3273293840.html (crack_and_chunk.py:127) [2025-02-27 07:31:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Laptop-Buyout-Request---FDT-Staff_3273293840.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Laptop-Recommendations-for-Division_3580624900.html (crack_and_chunk.py:127) [2025-02-27 07:31:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Laptop-Recommendations-for-Division_3580624900.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html (crack_and_chunk.py:127) [2025-02-27 07:31:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:22] INFO azureml.rag.embed - ==== Putting batch_id=4 with 100 chunks in queue (embed.py:320) [2025-02-27 07:31:26] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Embedded Documents - 400/500 documents_embedded (logging.py:383) [2025-02-27 07:31:27] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Total Documents - 400/500 documents_total (logging.py:383) [2025-02-27 07:31:27] INFO azureml.rag.chunk_embedder_0 - ==== embedding batch_id=4 with 100 chunks (embed.py:166) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Grafana-Dashboard_3491430402.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Grafana-Report---2024_3693477919.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Grafana-Report---2025_3810295862.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Grafana-Report_3693314067.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/GuardDuty-Findings-Join-Binance-AWS-Account_3761504260.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/HK-FortiGate-Vendor-Firewall-Comparison_3692920843.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/HK-Guest-Wi-Fi_3752722435.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/HK-Office-Network-Information_3260645500.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/HK-Office-Network-Information_3260645500.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/HK-Office-Network-Information_3260645500.html2 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/HKBN-Internet-Set-up-for-FortiGate-Firewall_3755704322.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hardware-purchase-from-vendor_2968846508.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hardware_3105357917.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Firewall-Log_3578757156.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Firewall-Memory-Issue_3569680563.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Firewall-Plan-Upgrade---Single-VDOM-Managing-Multiple-VLANs_3492806693.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Logs-Checking_3429171346.html (crack_and_chunk.py:127) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Firewall-Plan-Upgrade---Single-VDOM-Managing-Multiple-VLANs_3492806693.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Firewall-Plan-Upgrade---Single-VDOM-Managing-Multiple-VLANs_3492806693.html2 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Firewall-Upgrade-Proposal_3569680610.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Office-Floor-Plan_3583344643.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hong-Kong-Office_3260645470.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Hotlink-Protection_3777200142.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-To-Generate-HTML-Report-in-OWASP-ZAP_3813769264.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-To-Generate-Individual-HTML-Report-in-OWASP-ZAP_3814621195.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-To-Perform-An-Automated-Scan_3813572637.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-To-Perform-An-Manual-Scanning_3813539879.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-To-Scanning-Amazon-EC2-Instances-With-Amazon-Inspector_3457155076.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-To-Upgrade-WordPress-Version_3788505127.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-can-I-move-or-migrate-a-PRTG-installation-to-a-different-system-or-server_3740401754.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-Activate-an-eSIM-on-Company-iPhone_3753607177.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-Add-Apps-for-iOS-via-Microsoft-Intune_3358359557.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-Add-FD121-Email-Account-in-Windows_3819175946.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-Create-a-Google-Account-Using-Your-Existing-1stdigital-Email_3798007910.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-Make-Teams-App-Available-to-Everyone-in-Organization-using-Microsoft-Teams-Admin-Center_3638624257.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-Setup-Oauth-2.0-Configuration-for-Confluence_3769335810.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-dial-an-outside-number_3339714565.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/How-to-set-up-OneDrive-API_3777298477.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IAM-User-Access-Keys-To-Rotate_3180003341.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IPSec-Dial-up-Method-with-Firewall-Certificate-Authentication_3833790466.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IPsec-VPN---AllSecureVPN_3833102338.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ISMS-Board-of-Directors-Charter_3315335176.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ISMS-Scope_3244032039.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ISMS-Scope_3244032039.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IT-Admin-Procedures_2977628161.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IT-Disaster-Recovery-Plan_2909831169.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IT-Disaster-Recovery-Plan_2909831169.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IT-Policies_2880733197.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IT-Resilience_2880864264.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IT-Risk-Register---FD121_3500146689.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IT-Risk-Register---FDT_3425992930.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ITSM---JIRA-Service-Management-administration_2972778497.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/ITSM-User-Guide_2915434534.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Identity-Center-Azure-AD-SSO-for-TimesTrust-Account_3786637313.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Implement-TimesTrust-AWS-Account_3786473581.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Implementation-Steps_3516006405.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Incident-Management_3600220166.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Incident-Response-Procedures_3525640247.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Asset-Management_3558735939.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security-KRIs_3533602817.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security-Program_3342860289.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security-Program_3342860289.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security-Program_3342860289.html2 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security_2880864271.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security_2880864271.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security_2880864271.html2 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security_2880864271.html3 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Information-Security_2880864271.html4 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Installing-ModHeader-in-Your-Browser_3526000663.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Instructions-to-Create-a-Passkey-for-DocuSign-Users_3821273149.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Integration-of-AWS-Single-Sign-on-with-Azure-Active-Directory_3532587016.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Integration-of-AWS-Single-Sign-on-with-Azure-Active-Directory_3532587016.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/IoT-Devices-Policy_3538976884.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Ironscales---Email-Security_3821043734.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/January-9th-2025---25.1.102.1373_3834544136.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Jira-How-To-Remove-Label_3648258050.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Jira-Notification-Feed-Into-Microsoft-Teams-Channel_3620634626.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Jira-Reports-Tutorials_3646849036.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Join-DoIT-Organization-for-TimesTrust-Account_3786932226.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/KL-Guest-Wi-Fi_3590717447.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/KL-Meeting-Room-Video-Conference-Set-up_3582427186.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/KL-Menara-Prestige-Data-Ports_3580788740.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/KL-Menara-Prestige-Meeting-Room-VC-Diagram_3413409810.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/KL-Menara-Prestige-Network-Device-Information_3217522719.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/KL-Menara-Prestige-Network-Device-Information_3217522719.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/KRI_3693346853.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Kali-Linux_3790045231.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Kelvin---Research_3501391933.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Key-Rotation-on-AWS_3179773958.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Kibana-vs.-Grafana---A-Scenario-Based-Decision-Guide_3500179460.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Kibana-vs.-Grafana---A-Scenario-Based-Decision-Guide_3500179460.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Kibana-vs.-Grafana---A-Scenario-Based-Decision-Guide_3500179460.html2 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Knowledge-Base-Documentation_3442147346.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Laptop-Buyout-Request---FDT-Staff_3273293840.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Laptop-Buyout-Request---FDT-Staff_3273293840.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Laptop-Recommendations-for-Division_3580624900.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html0 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html1 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html2 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html3 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html4 (__init__.py:1300) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings - Documents to embed: 100 Documents reused: 0 (__init__.py:1337) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityStarted, Embeddings.embed (activity.py:108) [2025-02-27 07:31:27] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:28] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:31:28] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:31:28] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:31:29] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Logs-Checking_3429171346.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:29] INFO azureml.rag.crack_and_chunk - Processing file: TRM/MAS-GAP-Analysis-Assessment-Vendors_3547463701.html (crack_and_chunk.py:127) [2025-02-27 07:31:29] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:31:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:29] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:31:30] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 4 documents. (openai.py:196) [2025-02-27 07:31:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/MAS-GAP-Analysis-Assessment-Vendors_3547463701.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:32] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 370/370 processed_sources (logging.py:383) [2025-02-27 07:31:32] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 510/510 documents_total (logging.py:383) [2025-02-27 07:31:34] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:31:34] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Manage-Organizations_3612213249.html (crack_and_chunk.py:127) [2025-02-27 07:31:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:35] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Manage-Organizations_3612213249.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:35] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Management-Review_3288104994.html (crack_and_chunk.py:127) [2025-02-27 07:31:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:36] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Management-Review_3288104994.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:36] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Martin---Research_3729195009.html (crack_and_chunk.py:127) [2025-02-27 07:31:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Martin---Research_3729195009.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Menara-Prestige-Network-Topology_3256451091.html (crack_and_chunk.py:127) [2025-02-27 07:31:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Menara-Prestige-Network-Topology_3256451091.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:39] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Menara-Prestige-Office_3104800940.html (crack_and_chunk.py:127) [2025-02-27 07:31:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Menara-Prestige-Office_3104800940.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-365-Security-Center-at-First-Digital-Trust_3281682481.html (crack_and_chunk.py:127) [2025-02-27 07:31:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:43] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-365-Security-Center-at-First-Digital-Trust_3281682481.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:43] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Authenticator---Phone-Sign-in-process-Flowchart_3245670515.html (crack_and_chunk.py:127) [2025-02-27 07:31:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Authenticator---Phone-Sign-in-process-Flowchart_3245670515.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Authenticator-Phone-Sign-In-vs.-Microsoft-Passkey_3783622669.html (crack_and_chunk.py:127) [2025-02-27 07:31:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:46] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Authenticator-Phone-Sign-In-vs.-Microsoft-Passkey_3783622669.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Copilot-for-Office-365_3668803651.html (crack_and_chunk.py:127) [2025-02-27 07:31:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:46] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Copilot-for-Office-365_3668803651.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Defender-Vulnerability-Management_2968813896.html (crack_and_chunk.py:127) [2025-02-27 07:31:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Defender-Vulnerability-Management_2968813896.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:49] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 380/380 processed_sources (logging.py:383) [2025-02-27 07:31:51] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 520/520 documents_total (logging.py:383) [2025-02-27 07:31:52] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:31:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Defender-via-Microsoft-Intune_3282075655.html (crack_and_chunk.py:127) [2025-02-27 07:31:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:53] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Defender-via-Microsoft-Intune_3282075655.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:53] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Endpoint-Manager_2968846533.html (crack_and_chunk.py:127) [2025-02-27 07:31:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:55] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Endpoint-Manager_2968846533.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Entra-Internet-Access_3428974692.html (crack_and_chunk.py:127) [2025-02-27 07:31:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:55] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Entra-Internet-Access_3428974692.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Entra-Private-Access_3429171288.html (crack_and_chunk.py:127) [2025-02-27 07:31:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Entra-Private-Access_3429171288.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Guide-Document-Step-Configure_3429171250.html (crack_and_chunk.py:127) [2025-02-27 07:31:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:57] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Guide-Document-Step-Configure_3429171250.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:57] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Infrastructure_3429171232.html (crack_and_chunk.py:127) [2025-02-27 07:31:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:59] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Infrastructure_3429171232.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:31:59] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Intune-Scripts_3690135564.html (crack_and_chunk.py:127) [2025-02-27 07:31:59] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:31:59] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:01] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Intune-Scripts_3690135564.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:01] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Microsoft-Reports-2024_3706617857.html (crack_and_chunk.py:127) [2025-02-27 07:32:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:01] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:02] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Microsoft-Reports-2024_3706617857.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:02] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Mimecast---Email-Security_3840966669.html (crack_and_chunk.py:127) [2025-02-27 07:32:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:04] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Mimecast---Email-Security_3840966669.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:04] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Minimum-TLS-Version_3776839757.html (crack_and_chunk.py:127) [2025-02-27 07:32:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Minimum-TLS-Version_3776839757.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:07] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 390/390 processed_sources (logging.py:383) [2025-02-27 07:32:08] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 531/531 documents_total (logging.py:383) [2025-02-27 07:32:09] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:32:09] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Mission-and-Objectives_3515842583.html (crack_and_chunk.py:127) [2025-02-27 07:32:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Mission-and-Objectives_3515842583.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Mod-Header-Add-Request-URL-Filter_3612967013.html (crack_and_chunk.py:127) [2025-02-27 07:32:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:11] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:12] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Mod-Header-Add-Request-URL-Filter_3612967013.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Monitoring-and-Detection-Procedures_3515023396.html (crack_and_chunk.py:127) [2025-02-27 07:32:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:14] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Monitoring-and-Detection-Procedures_3515023396.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:14] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Monitoring_3172630533.html (crack_and_chunk.py:127) [2025-02-27 07:32:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Monitoring_3172630533.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Moved-shortcuts_3483041795.html (crack_and_chunk.py:127) [2025-02-27 07:32:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Moved-shortcuts_3483041795.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:17] INFO azureml.rag.crack_and_chunk - Processing file: TRM/NMAP---Network-Scanning-and-Enumeration_3791159302.html (crack_and_chunk.py:127) [2025-02-27 07:32:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/NMAP---Network-Scanning-and-Enumeration_3791159302.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/NMAP---Zenmap_3796828161.html (crack_and_chunk.py:127) [2025-02-27 07:32:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:19] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/NMAP---Zenmap_3796828161.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Nabil---Research_3501391923.html (crack_and_chunk.py:127) [2025-02-27 07:32:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:21] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Nabil---Research_3501391923.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:21] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Network-architectures_3247145187.html (crack_and_chunk.py:127) [2025-02-27 07:32:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Network-architectures_3247145187.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Network_3105357892.html (crack_and_chunk.py:127) [2025-02-27 07:32:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:24] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Network_3105357892.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:25] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 400/400 processed_sources (logging.py:383) [2025-02-27 07:32:26] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 541/541 documents_total (logging.py:383) [2025-02-27 07:32:27] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:32:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Notification-Jira-Cloud-with-Microsoft-Teams_3591929892.html (crack_and_chunk.py:127) [2025-02-27 07:32:28] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityCompleted: Activity=Embeddings.embed, HowEnded=Success, Duration=60940.42 [ms] (activity.py:129) [2025-02-27 07:32:28] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:32:28] INFO azureml.rag.chunk_embedder_0 - Embedding took 61.010363817214966 seconds (embed.py:184) [2025-02-27 07:32:28] INFO azureml.rag.chunk_embedder_0 - Only data will be saved (embed.py:200) [2025-02-27 07:32:28] INFO azureml.rag.chunk_embedder_0 - waiting for chunk_batch: 0 (embed.py:159) [2025-02-27 07:32:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:28] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Notification-Jira-Cloud-with-Microsoft-Teams_3591929892.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/November-12th-2024---Version-24.4.102.1351_3769434173.html (crack_and_chunk.py:127) [2025-02-27 07:32:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/November-12th-2024---Version-24.4.102.1351_3769434173.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/OWASP-ZAP-Attack-Proxy_3793485847.html (crack_and_chunk.py:127) [2025-02-27 07:32:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/OWASP-ZAP-Attack-Proxy_3793485847.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:31] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Offboarding-FDT-Staff-Procedure-for-IT-Department_3074949157.html (crack_and_chunk.py:127) [2025-02-27 07:32:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Offboarding-FDT-Staff-Procedure-for-IT-Department_3074949157.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:32] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Onboarding-Procedure-for-New-Employees_2970288129.html (crack_and_chunk.py:127) [2025-02-27 07:32:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:34] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Onboarding-Procedure-for-New-Employees_2970288129.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:34] INFO azureml.rag.crack_and_chunk - Processing file: TRM/OpenVAS---Vulnerability-Scan_3798695938.html (crack_and_chunk.py:127) [2025-02-27 07:32:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:36] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/OpenVAS---Vulnerability-Scan_3798695938.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:36] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Operational-Processes_3515940872.html (crack_and_chunk.py:127) [2025-02-27 07:32:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:37] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Operational-Processes_3515940872.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:37] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Network-Monitor-Information_3509288994.html (crack_and_chunk.py:127) [2025-02-27 07:32:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Network-Monitor-Information_3509288994.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:39] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Network-Monitor-Mobile-Application_3511025678.html (crack_and_chunk.py:127) [2025-02-27 07:32:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:40] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Network-Monitor-Mobile-Application_3511025678.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Network-Monitor-Received-Notification_3511058447.html (crack_and_chunk.py:127) [2025-02-27 07:32:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:40] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Network-Monitor-Received-Notification_3511058447.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:43] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 410/410 processed_sources (logging.py:383) [2025-02-27 07:32:44] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 556/556 documents_total (logging.py:383) [2025-02-27 07:32:45] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:32:45] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Network-Monitor-VPN-Tunnels-FortiGate_3510403073.html (crack_and_chunk.py:127) [2025-02-27 07:32:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Network-Monitor-VPN-Tunnels-FortiGate_3510403073.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Network-Monitor_3499229248.html (crack_and_chunk.py:127) [2025-02-27 07:32:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Network-Monitor_3499229248.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Network-Monitoring_3532259352.html (crack_and_chunk.py:127) [2025-02-27 07:32:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:48] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Network-Monitoring_3532259352.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:48] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Re-Search-Information_3685187589.html (crack_and_chunk.py:127) [2025-02-27 07:32:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:49] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Re-Search-Information_3685187589.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Release-Notes_3666378753.html (crack_and_chunk.py:127) [2025-02-27 07:32:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Release-Notes_3666378753.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Report---2024_3685122056.html (crack_and_chunk.py:127) [2025-02-27 07:32:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:52] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:52] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Report---2024_3685122056.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:52] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Report---2025_3810426928.html (crack_and_chunk.py:127) [2025-02-27 07:32:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Report---2025_3810426928.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Report_3681878020.html (crack_and_chunk.py:127) [2025-02-27 07:32:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:54] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:56] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Report_3681878020.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:56] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG-Teams-Notification-Channel_3509321734.html (crack_and_chunk.py:127) [2025-02-27 07:32:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:56] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:57] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG-Teams-Notification-Channel_3509321734.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:32:57] INFO azureml.rag.crack_and_chunk - Processing file: TRM/PRTG_3740663834.html (crack_and_chunk.py:127) [2025-02-27 07:32:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:32:59] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/PRTG_3740663834.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:00] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 420/420 processed_sources (logging.py:383) [2025-02-27 07:33:02] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 568/568 documents_total (logging.py:383) [2025-02-27 07:33:03] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:33:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html (crack_and_chunk.py:127) [2025-02-27 07:33:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:03] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:03] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:03] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Patch-Management_3600449539.html (crack_and_chunk.py:127) [2025-02-27 07:33:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:04] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Patch-Management_3600449539.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Penetration-Testing-Services_3480846420.html (crack_and_chunk.py:127) [2025-02-27 07:33:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:05] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:05] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Penetration-Testing-Services_3480846420.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:05] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Pentest-Tools_3790209040.html (crack_and_chunk.py:127) [2025-02-27 07:33:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:07] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Pentest-Tools_3790209040.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:07] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Physical-Security_3197894703.html (crack_and_chunk.py:127) [2025-02-27 07:33:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:07] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Physical-Security_3197894703.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Pre-Defined-Thresholds-Reached-In-Grafana_3519807502.html (crack_and_chunk.py:127) [2025-02-27 07:33:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:12] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Pre-Defined-Thresholds-Reached-In-Grafana_3519807502.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:12] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Printer_3110174721.html (crack_and_chunk.py:127) [2025-02-27 07:33:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:14] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Printer_3110174721.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:14] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Problem-Management_3600351235.html (crack_and_chunk.py:127) [2025-02-27 07:33:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:14] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Problem-Management_3600351235.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Project-Management_2883158017.html (crack_and_chunk.py:127) [2025-02-27 07:33:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:16] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Project-Management_2883158017.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Proofpoint---Email-Security_3831332910.html (crack_and_chunk.py:127) [2025-02-27 07:33:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:16] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:17] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Proofpoint---Email-Security_3831332910.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:19] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 430/430 processed_sources (logging.py:383) [2025-02-27 07:33:20] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 586/586 documents_total (logging.py:383) [2025-02-27 07:33:21] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:33:21] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Purchase-Saving-Plan-for-TimesTrust-Account_3786833986.html (crack_and_chunk.py:127) [2025-02-27 07:33:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:22] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:23] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Purchase-Saving-Plan-for-TimesTrust-Account_3786833986.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:23] INFO azureml.rag.crack_and_chunk - Processing file: TRM/QA-Process_3600613377.html (crack_and_chunk.py:127) [2025-02-27 07:33:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:23] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:25] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/QA-Process_3600613377.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:25] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Quick-Start_3104014341.html (crack_and_chunk.py:127) [2025-02-27 07:33:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:25] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:26] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Quick-Start_3104014341.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:26] INFO azureml.rag.crack_and_chunk - Processing file: TRM/RDS-Instance-Upgrade-for-TimesTrust-Account_3786604548.html (crack_and_chunk.py:127) [2025-02-27 07:33:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:27] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/RDS-Instance-Upgrade-for-TimesTrust-Account_3786604548.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Regulatory-specific-IT-Policies_3554246720.html (crack_and_chunk.py:127) [2025-02-27 07:33:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:28] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Regulatory-specific-IT-Policies_3554246720.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:28] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Remote-Work-Equipment-Guidelines_3791847482.html (crack_and_chunk.py:127) [2025-02-27 07:33:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Remote-Work-Equipment-Guidelines_3791847482.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Remote-Worker-Assets-Procedure_3791847501.html (crack_and_chunk.py:127) [2025-02-27 07:33:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:30] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:31] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Remote-Worker-Assets-Procedure_3791847501.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:31] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Resetting-FortiGate-Firewall-to-Factory-Settings_3765469187.html (crack_and_chunk.py:127) [2025-02-27 07:33:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:33] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Resetting-FortiGate-Firewall-to-Factory-Settings_3765469187.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:33] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Retrieve-a-HAR-file_3551002636.html (crack_and_chunk.py:127) [2025-02-27 07:33:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:33] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:34] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Retrieve-a-HAR-file_3551002636.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:34] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Ring-Order-in-Phone-Systems_3160145970.html (crack_and_chunk.py:127) [2025-02-27 07:33:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:36] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Ring-Order-in-Phone-Systems_3160145970.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:37] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 440/440 processed_sources (logging.py:383) [2025-02-27 07:33:38] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 597/597 documents_total (logging.py:383) [2025-02-27 07:33:40] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:33:40] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Risk-Assessment_3245572183.html (crack_and_chunk.py:127) [2025-02-27 07:33:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:41] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:42] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Risk-Assessment_3245572183.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:42] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Roles-and-permissions_3602448404.html (crack_and_chunk.py:127) [2025-02-27 07:33:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:43] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Roles-and-permissions_3602448404.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:43] INFO azureml.rag.embed - ==== Putting batch_id=5 with 100 chunks in queue (embed.py:320) [2025-02-27 07:33:47] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Embedded Documents - 500/600 documents_embedded (logging.py:383) [2025-02-27 07:33:48] INFO azureml.rag.crack_and_chunk_and_embed.create_embeddings - Status: embedding - Total Documents - 500/600 documents_total (logging.py:383) [2025-02-27 07:33:48] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Route53-Private-Hosted-Zones_3484778578.html (crack_and_chunk.py:127) [2025-02-27 07:33:48] INFO azureml.rag.chunk_embedder_1 - ==== embedding batch_id=5 with 100 chunks (embed.py:166) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html5 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html6 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html7 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html8 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html9 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html10 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html11 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/List-of-the-settings-in-the-Windows-365-Cloud-PC-security-baseline-in-Intune_3627319301.html12 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Logs-Checking_3429171346.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/MAS-GAP-Analysis-Assessment-Vendors_3547463701.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Manage-Organizations_3612213249.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Management-Review_3288104994.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Martin---Research_3729195009.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Menara-Prestige-Network-Topology_3256451091.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Menara-Prestige-Office_3104800940.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-365-Security-Center-at-First-Digital-Trust_3281682481.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Authenticator---Phone-Sign-in-process-Flowchart_3245670515.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Authenticator-Phone-Sign-In-vs.-Microsoft-Passkey_3783622669.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Copilot-for-Office-365_3668803651.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Defender-Vulnerability-Management_2968813896.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Defender-via-Microsoft-Intune_3282075655.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Endpoint-Manager_2968846533.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Entra-Internet-Access_3428974692.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Entra-Private-Access_3429171288.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Entra-Private-Access_3429171288.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Guide-Document-Step-Configure_3429171250.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Infrastructure_3429171232.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Intune-Scripts_3690135564.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Microsoft-Reports-2024_3706617857.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Mimecast---Email-Security_3840966669.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Minimum-TLS-Version_3776839757.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Mission-and-Objectives_3515842583.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Mod-Header-Add-Request-URL-Filter_3612967013.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Monitoring-and-Detection-Procedures_3515023396.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Monitoring_3172630533.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Moved-shortcuts_3483041795.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/NMAP---Network-Scanning-and-Enumeration_3791159302.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/NMAP---Zenmap_3796828161.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Nabil---Research_3501391923.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Network-architectures_3247145187.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Network_3105357892.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Notification-Jira-Cloud-with-Microsoft-Teams_3591929892.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/November-12th-2024---Version-24.4.102.1351_3769434173.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/November-12th-2024---Version-24.4.102.1351_3769434173.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/November-12th-2024---Version-24.4.102.1351_3769434173.html2 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/November-12th-2024---Version-24.4.102.1351_3769434173.html3 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/November-12th-2024---Version-24.4.102.1351_3769434173.html4 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/OWASP-ZAP-Attack-Proxy_3793485847.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/OWASP-ZAP-Attack-Proxy_3793485847.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Offboarding-FDT-Staff-Procedure-for-IT-Department_3074949157.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Onboarding-Procedure-for-New-Employees_2970288129.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/OpenVAS---Vulnerability-Scan_3798695938.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Operational-Processes_3515940872.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Network-Monitor-Information_3509288994.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Network-Monitor-Mobile-Application_3511025678.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Network-Monitor-Received-Notification_3511058447.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Network-Monitor-VPN-Tunnels-FortiGate_3510403073.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Network-Monitor_3499229248.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Network-Monitor_3499229248.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Network-Monitoring_3532259352.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Re-Search-Information_3685187589.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Release-Notes_3666378753.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Report---2024_3685122056.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Report---2024_3685122056.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Report---2025_3810426928.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Report_3681878020.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG-Teams-Notification-Channel_3509321734.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/PRTG_3740663834.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html2 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html3 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html4 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html5 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Parameter-Store-Parameters-To-Rotate-Or-Delete_3208052742.html6 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Patch-Management_3600449539.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Penetration-Testing-Services_3480846420.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Penetration-Testing-Services_3480846420.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Pentest-Tools_3790209040.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Physical-Security_3197894703.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Pre-Defined-Thresholds-Reached-In-Grafana_3519807502.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Printer_3110174721.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Problem-Management_3600351235.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Project-Management_2883158017.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Project-Management_2883158017.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Proofpoint---Email-Security_3831332910.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Purchase-Saving-Plan-for-TimesTrust-Account_3786833986.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/QA-Process_3600613377.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Quick-Start_3104014341.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/RDS-Instance-Upgrade-for-TimesTrust-Account_3786604548.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/RDS-Instance-Upgrade-for-TimesTrust-Account_3786604548.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Regulatory-specific-IT-Policies_3554246720.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Remote-Work-Equipment-Guidelines_3791847482.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Remote-Worker-Assets-Procedure_3791847501.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Resetting-FortiGate-Firewall-to-Factory-Settings_3765469187.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Retrieve-a-HAR-file_3551002636.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Ring-Order-in-Phone-Systems_3160145970.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Risk-Assessment_3245572183.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Roles-and-permissions_3602448404.html0 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Processing document: TRM/Roles-and-permissions_3602448404.html1 (__init__.py:1300) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings - Documents to embed: 100 Documents reused: 0 (__init__.py:1337) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityStarted, Embeddings.embed (activity.py:108) [2025-02-27 07:33:48] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:49] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:33:49] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:33:49] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:33:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Route53-Private-Hosted-Zones_3484778578.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Route53-Private-Self-Signed-SSL-Certificate-Generate_3495297026.html (crack_and_chunk.py:127) [2025-02-27 07:33:50] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:33:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:50] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:50] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 16 documents. (openai.py:196) [2025-02-27 07:33:53] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Route53-Private-Self-Signed-SSL-Certificate-Generate_3495297026.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:53] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Route53-Private-Sub-Domain-Listing-Information_3491921922.html (crack_and_chunk.py:127) [2025-02-27 07:33:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:53] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Route53-Private-Sub-Domain-Listing-Information_3491921922.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:53] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SIP-Phone-Network-Topology_3245572224.html (crack_and_chunk.py:127) [2025-02-27 07:33:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:53] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:54] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SIP-Phone-Network-Topology_3245572224.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:54] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SOC-2---Description-of-Controls_2896920624.html (crack_and_chunk.py:127) [2025-02-27 07:33:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:55] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SOC-2---Description-of-Controls_2896920624.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:55] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SOC-II-Auditors-Research_3516071939.html (crack_and_chunk.py:127) [2025-02-27 07:33:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:55] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:58] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SOC-II-Auditors-Research_3516071939.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:58] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SOC-Microsoft-Components-and-Tools_3515023386.html (crack_and_chunk.py:127) [2025-02-27 07:33:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:58] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:33:59] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SOC-Microsoft-Components-and-Tools_3515023386.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:33:59] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SOC-Roles-and-Responsibilities_3514925078.html (crack_and_chunk.py:127) [2025-02-27 07:34:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:00] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:01] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SOC-Roles-and-Responsibilities_3514925078.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:02] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 450/450 processed_sources (logging.py:383) [2025-02-27 07:34:04] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 612/612 documents_total (logging.py:383) [2025-02-27 07:34:06] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:34:06] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SSL-Cert-Expired-Soon-for-TimesTrust.com_3786801180.html (crack_and_chunk.py:127) [2025-02-27 07:34:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:06] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:08] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SSL-Cert-Expired-Soon-for-TimesTrust.com_3786801180.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:08] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SSL-VPN-Fortinet-Firewall-Policies_3584983045.html (crack_and_chunk.py:127) [2025-02-27 07:34:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:08] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:10] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SSL-VPN-Fortinet-Firewall-Policies_3584983045.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:10] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SSL-VPN-Network-Diagram_3585212417.html (crack_and_chunk.py:127) [2025-02-27 07:34:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:10] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:11] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SSL-VPN-Network-Diagram_3585212417.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:11] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SSL-VPN-for-FD121-Portal_3417473043.html (crack_and_chunk.py:127) [2025-02-27 07:34:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:12] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:13] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SSL-VPN-for-FD121-Portal_3417473043.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:13] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SSL-VPN-vs-IPSEC-VPN_3835101199.html (crack_and_chunk.py:127) [2025-02-27 07:34:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:13] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:15] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SSL-VPN-vs-IPSEC-VPN_3835101199.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:15] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SSL-VPN_3243638821.html (crack_and_chunk.py:127) [2025-02-27 07:34:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:15] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:16] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SSL-VPN_3243638821.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:16] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Scanning-An-Authenticated-Web-Application_3815866416.html (crack_and_chunk.py:127) [2025-02-27 07:34:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:17] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:18] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Scanning-An-Authenticated-Web-Application_3815866416.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:18] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Secrets-Key-Value-Pairs_3223289869.html (crack_and_chunk.py:127) [2025-02-27 07:34:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:18] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:19] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Secrets-Key-Value-Pairs_3223289869.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:19] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Secure-Code-Warrior_3522789626.html (crack_and_chunk.py:127) [2025-02-27 07:34:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:20] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:20] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Secure-Code-Warrior_3522789626.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:20] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Secure-Collaboration-on-Miro-with-External-Parties_3307438082.html (crack_and_chunk.py:127) [2025-02-27 07:34:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:21] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:22] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Secure-Collaboration-on-Miro-with-External-Parties_3307438082.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:24] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 460/460 processed_sources (logging.py:383) [2025-02-27 07:34:26] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 622/622 documents_total (logging.py:383) [2025-02-27 07:34:27] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:34:27] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Secure-System-Architecture-and-Engineering-Principles_3240296477.html (crack_and_chunk.py:127) [2025-02-27 07:34:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:27] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:29] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Secure-System-Architecture-and-Engineering-Principles_3240296477.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:29] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Securing-AWS-Console-Access-with-IP-Whitelisting_3740401721.html (crack_and_chunk.py:127) [2025-02-27 07:34:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:29] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:30] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Securing-AWS-Console-Access-with-IP-Whitelisting_3740401721.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:30] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Security-Baseline_3627384839.html (crack_and_chunk.py:127) [2025-02-27 07:34:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:31] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:32] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Security-Baseline_3627384839.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:32] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Security-Group-No-Filter-and-Allow-All-for-TimesTrust-Account_3786604643.html (crack_and_chunk.py:127) [2025-02-27 07:34:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:32] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:33] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Security-Group-No-Filter-and-Allow-All-for-TimesTrust-Account_3786604643.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:33] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Self-Signed-and-Let%27s-Encrypt-Certificates_3499524101.html (crack_and_chunk.py:127) [2025-02-27 07:34:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:34] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Self-Signed-and-Let%27s-Encrypt-Certificates_3499524101.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:34] INFO azureml.rag.crack_and_chunk - Processing file: TRM/September-25th-2024---Version-24.3.100.1361_3666313223.html (crack_and_chunk.py:127) [2025-02-27 07:34:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:34] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:34] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/September-25th-2024---Version-24.3.100.1361_3666313223.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:34] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Site-To-Site-VPN-Link-With-FortiGate_3526426629.html (crack_and_chunk.py:127) [2025-02-27 07:34:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:35] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:36] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Site-To-Site-VPN-Link-With-FortiGate_3526426629.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:36] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Site-to-Site-VPN-Status-Down-for-TimesTrust_3787030590.html (crack_and_chunk.py:127) [2025-02-27 07:34:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:36] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Site-to-Site-VPN-Status-Down-for-TimesTrust_3787030590.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Slack-Channel-to-Teams-Channel_3567681544.html (crack_and_chunk.py:127) [2025-02-27 07:34:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:38] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:38] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Slack-Channel-to-Teams-Channel_3567681544.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:38] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Slack-License-Revoke_3606904835.html (crack_and_chunk.py:127) [2025-02-27 07:34:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:39] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:39] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Slack-License-Revoke_3606904835.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:40] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Processed Sources - 470/470 processed_sources (logging.py:383) [2025-02-27 07:34:41] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Total Documents - 639/639 documents_total (logging.py:383) [2025-02-27 07:34:43] INFO azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - Status: sourcing - Reused Documents - 0/0 documents_reused (logging.py:383) [2025-02-27 07:34:43] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Smart-Contract-Audit-Firm-Research_3264839984.html (crack_and_chunk.py:127) [2025-02-27 07:34:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:43] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Smart-Contract-Audit-Firm-Research_3264839984.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:43] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Smart-Trust-Release-Management_3600711681.html (crack_and_chunk.py:127) [2025-02-27 07:34:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:43] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:44] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Smart-Trust-Release-Management_3600711681.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:44] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SmartTrust-Backoffice-Prod_3355050000.html (crack_and_chunk.py:127) [2025-02-27 07:34:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:45] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:46] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SmartTrust-Backoffice-Prod_3355050000.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:46] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Snyk-vs-SonarQube-Cloud-Comparison_3809116189.html (crack_and_chunk.py:127) [2025-02-27 07:34:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:47] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Snyk-vs-SonarQube-Cloud-Comparison_3809116189.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:47] INFO azureml.rag.crack_and_chunk - Processing file: TRM/Software_3105259616.html (crack_and_chunk.py:127) [2025-02-27 07:34:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:47] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:48] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/Software_3105259616.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:48] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SonarCloud-Project-Responsibility-Person-In-Charges_3490021385.html (crack_and_chunk.py:127) [2025-02-27 07:34:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:48] INFO azureml.rag.azureml.rag.documents.chunking - Using HTML splitter. (chunking.py:109) [2025-02-27 07:34:50] INFO azureml.rag.crack_and_chunk_and_embed - Processing chunks for source: TRM/SonarCloud-Project-Responsibility-Person-In-Charges_3490021385.html (crack_and_chunk_and_embed.py:219) [2025-02-27 07:34:50] INFO azureml.rag.crack_and_chunk - Processing file: TRM/SonarCloud-and-AWS-Inspector_3442180137.html (crack_and_chunk.py:127) [2025-02-27 07:34:50] INFO azureml.rag.embeddings.openai - Attempt 0 to embed 4 documents. (openai.py:196) [2025-02-27 07:34:50] INFO azureml.rag.azureml.rag.embeddings.Embeddings.embed - ActivityCompleted: Activity=Embeddings.embed, HowEnded=Success, Duration=61764.81 [ms] (activity.py:129) [2025-02-27 07:34:50] INFO azureml.rag.connections - The connection 'kshe-m7llzpu3-eastus2_aoai' is a with api_key auth type. (connections.py:184) [2025-02-27 07:34:50] INFO azureml.rag.chunk_embedder_1 - Embedding took 61.83719444274902 seconds (embed.py:184) [2025-02-27 07:34:50] INFO azureml.rag.chunk_embedder_1 - Only data will be saved (embed.py:200) [2025-02-27 07:34:50] INFO azureml.rag.chunk_embedder_1 - waiting for chunk_batch: 1 (embed.py:159) [2025-02-27 07:34:50] ERROR azureml.rag.crack_and_chunk_and_embed.create_embeddings - ActivityCompleted: Activity=create_embeddings, HowEnded=Failure, Duration=943686.89 [ms], Exception=UnicodeDecodeError (activity.py:127) [2025-02-27 07:34:50] ERROR azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - ServiceError: intepreted error = Rag system error, original error = 'gb2312' codec can't decode byte 0xf0 in position 811: illegal multibyte sequence (exceptions.py:124) [2025-02-27 07:34:55] ERROR azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - crack_and_chunk failed with exception: Traceback (most recent call last): File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 506, in main_wrapper map_exceptions(main, activity_logger, args, logger, activity_logger) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/utils/exceptions.py", line 126, in map_exceptions raise e File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/utils/exceptions.py", line 118, in map_exceptions return func(*func_args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 475, in main embeddings_container = crack_and_chunk_and_embed( File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 344, in crack_and_chunk_and_embed num_embedded = create_embeddings( File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/embed.py", line 312, in create_embeddings for chunk in chunks: File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 218, in documents_to_embed for chunked_doc in chunked_docs: File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/chunking.py", line 169, in split_documents for i, document in enumerate(documents): File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 376, in crack_documents raise e File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 365, in crack_documents yield loader.load_chunked_document() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 71, in load_chunked_document pages = self.load() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 132, in load docs = super().load() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/langchain/vendor/document_loaders/unstructured.py", line 79, in load elements = self._get_elements() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 148, in _get_elements return partition_html(file=self.file, **self.unstructured_kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/documents/elements.py", line 605, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/filetype.py", line 706, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/filetype.py", line 662, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/chunking/dispatch.py", line 74, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 103, in partition_html elements = list( File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/lang.py", line 475, in apply_lang_metadata elements = list(elements) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 222, in iter_elements yield from cls(opts)._iter_elements() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 229, in _iter_elements for e in self._main.iter_elements(): File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/utils.py", line 155, in __get__ value = self._fget(obj) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 239, in _main html_text = self._opts.html_text File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/utils.py", line 155, in __get__ value = self._fget(obj) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 164, in html_text return read_txt_file(file=self._file, encoding=self._encoding)[1] File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/encoding.py", line 136, in read_txt_file formatted_encoding, file_text = detect_file_encoding(file=file) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/encoding.py", line 101, in detect_file_encoding file_text = byte_data.decode(encoding) UnicodeDecodeError: 'gb2312' codec can't decode byte 0xf0 in position 811: illegal multibyte sequence (crack_and_chunk_and_embed.py:508) [2025-02-27 07:34:55] ERROR azureml.rag.crack_and_chunk_and_embed.crack_and_chunk_and_embed - ActivityCompleted: Activity=crack_and_chunk_and_embed, HowEnded=Failure, Duration=951847.24 [ms], Exception=UnicodeDecodeError (activity.py:127) Traceback (most recent call last): File "/azureml-envs/rag-embeddings/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/azureml-envs/rag-embeddings/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 559, in main_wrapper(args, logger) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 511, in main_wrapper raise e File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 506, in main_wrapper map_exceptions(main, activity_logger, args, logger, activity_logger) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/utils/exceptions.py", line 126, in map_exceptions raise e File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/utils/exceptions.py", line 118, in map_exceptions return func(*func_args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 475, in main embeddings_container = crack_and_chunk_and_embed( File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 344, in crack_and_chunk_and_embed num_embedded = create_embeddings( File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/embed.py", line 312, in create_embeddings for chunk in chunks: File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/tasks/crack_and_chunk_and_embed.py", line 218, in documents_to_embed for chunked_doc in chunked_docs: File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/chunking.py", line 169, in split_documents for i, document in enumerate(documents): File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 376, in crack_documents raise e File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 365, in crack_documents yield loader.load_chunked_document() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 71, in load_chunked_document pages = self.load() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 132, in load docs = super().load() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/langchain/vendor/document_loaders/unstructured.py", line 79, in load elements = self._get_elements() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/azureml/rag/documents/cracking.py", line 148, in _get_elements return partition_html(file=self.file, **self.unstructured_kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/documents/elements.py", line 605, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/filetype.py", line 706, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/filetype.py", line 662, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/chunking/dispatch.py", line 74, in wrapper elements = func(*args, **kwargs) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 103, in partition_html elements = list( File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/lang.py", line 475, in apply_lang_metadata elements = list(elements) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 222, in iter_elements yield from cls(opts)._iter_elements() File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 229, in _iter_elements for e in self._main.iter_elements(): File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/utils.py", line 155, in __get__ value = self._fget(obj) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 239, in _main html_text = self._opts.html_text File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/utils.py", line 155, in __get__ value = self._fget(obj) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/partition/html/partition.py", line 164, in html_text return read_txt_file(file=self._file, encoding=self._encoding)[1] File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/encoding.py", line 136, in read_txt_file formatted_encoding, file_text = detect_file_encoding(file=file) File "/azureml-envs/rag-embeddings/lib/python3.9/site-packages/unstructured/file_utils/encoding.py", line 101, in detect_file_encoding file_text = byte_data.decode(encoding) UnicodeDecodeError: 'gb2312' codec can't decode byte 0xf0 in position 811: illegal multibyte sequence