# Converters test guidance — GisConverter.Lib.Converters

This folder contains tests and documentation that validate converter implementations and the helper logic that supports them. Use this README when adding or debugging tests for detection heuristics, archive handling, orchestrator logic and converter wiring.

## Overview
- Purpose: provide deterministic unit tests for detection and routing, small gated integration tests for end-to-end conversions, and logging/diagnostic tests to aid triage.
- Scope: JSON and archive detection heuristics, ConverterFactory routing, ConverterUtils helpers (safe extraction, zip-slip guards, output path construction), UniversalGisConverter orchestration, and lightweight adapters/fakes used by tests.

## Principles
- Keep unit tests fast, deterministic and isolated. Use fakes and injected registration maps to avoid heavy dependencies in unit tests.
- Gate heavy or licensed integration tests so main CI remains fast and usable for contributors without private samples or licenses.
- Prefer tolerant assertions (case-insensitive substring checks, presence checks) over brittle exact-string assertions.

---

## Test categories

1. Unit tests
   - Validate detection heuristics (JSON variants, header sniffing, NDJSON), factory routing and `ConverterUtils` helpers (zip-slip, path validation).
   - Use injected registration dictionaries or a `FakeFactory` that records `LastRequestedKey` so tests can assert which converter the detection attempted to resolve.

2. Integration tests (gated)
   - End-to-end conversions that may exercise third-party drivers (for example Aspose.GIS). Keep sample artifacts small and gate these tests with environment variables and/or license checks.
   - Place sample artifacts under `TestsApp/TestData/<format>` and mark them `CopyToOutputDirectory=PreserveNewest` so they are discoverable via `AppContext.BaseDirectory`.

3. Logging & diagnostics tests
   - Capture logs via `TestLogger` and assert with `LogAssert` helpers. Assert tokens at the expected `LogLevel` rather than full messages.

---

## Recommended patterns & helpers

- Factory injection
  - For unit tests create a small `Dictionary<string, Func<IConverter>>` and pass it to `new ConverterFactory(registrations, testLogger)`. This keeps tests fast and deterministic.
  - Provide a `FakeFactory` or `DummyConverter` local to tests that returns `ConversionResult.Success("ok")` so detection flow proceeds when needed by caller APIs.

- Temporary file hygiene
  - Use a GUID-based temp root:
    `Path.Combine(Path.GetTempPath(), "GisConverter.<suite>", Guid.NewGuid().ToString("N"))`.
  - Clean up in `Dispose()` using best-effort deletion wrapped in try/catch and log cleanup failures to Debug.
  - Create per-target `out` and `tmp` subfolders and delete them in finally blocks to minimize artifacts on CI agents.

- Archive handling
  - Use `ZipFile.CreateFromDirectory` or `ZipArchive` to construct small deterministic archives.
  - When preserving folder segments in zips is required (for example `.gdb` folders), create the archive with `includeBaseDirectory:true` so entries include the base folder name.
  - For detection tests prefer enumerating entries and reading a bounded prefix rather than extracting the archive to disk when possible.

- JSON detection
  - Provide minimal canonical snippets for GeoJSON, EsriJSON, TopoJSON and GeoJSON sequences (NDJSON). Include edge cases: BOMs, leading whitespace, truncated JSON and conflicting markers.
  - For NDJSON detection include tests with and without BOM and with extra blank lines.

- Assertions
  - Prefer stable checks:
    - `ConversionResult.Status` equality
    - Existence/non-empty checks for output files (avoid exact filename assertions)
    - Case-insensitive substring checks on `ConversionResult.Message` and logs
  - Use `FileExtensionHelpers.FromOption(...)` and `ToDotExtension(...)` in assertions to avoid hard-coded extensions.

---

## Integration gating & licensing
- Gate integration tests with environment variables and license checks to keep public CI green:
  - `GISCONVERTER_ENABLE_INTEGRATION=1` to run gated integration tests.
  - `GISCONVERTER_ASPOSE_LICENSE_PATH` (or similar) to provide Aspose license for tests that require it.
- Integration tests should return early (skip) when sample data or licenses are not available.

---

## Key logs & tokens to assert
- Parameter trace (Debug): `CsvConverter.Convert params:` — used for diagnostic parameter capture.
- Conversion start (Info): `starting conversion (option='...')`.
- Conversion success (Info): `Conversion succeeded: <path>`.
- Conversion failure (Error/Info): `Conversion failed: <message>`.
- Cleanup lifecycle (Debug/Info/Warn): `Cleaning up temp folder.`, `Temp folder '<path>' deleted.`, `Cleanup failed:`.

---

## When to update tests
- Adding a new supported target format:
  - Update `IntegrationTestConstants.TargetOptions` (TestsApp) and add minimal integration samples and gated tests.
  - Add mapping entries in `FileExtensionHelpers` and update `ConverterUtils` mapping if driver behavior differs.

- Changing detector heuristics:
  - Update detection tests (JsonFormatDetectorTests, ConverterFactoryInputExtensionsTests) with new expected tokens and include rationale in the PR description so reviewers can validate the change.

---

## CI guidance
- Keep unit tests fast and free of external dependencies.
- Run licensed integration scenarios in separate pipeline jobs that provision the required secrets and sample artifacts.
- Store licenses and private samples securely (pipeline secrets, protected storage). Do not commit any license files to the repository.

---

## Troubleshooting checklist
- "No Program type found" errors in CLI tests:
  - Ensure `GisConverter.ConsoleApp` is built and the assembly is present in the test probe locations. The tests that use reflection list the probed paths — inspect them to debug.

- Logging assertion failures:
  - Dump `TestLogger` contents to test output and inspect captured tokens and levels. Prefer updating assertions to check stable tokens rather than full messages.

- Archive extraction failures:
  - Verify your test-created zips contain the expected entry names and folder segments. Use `includeBaseDirectory` when needed.

---

## Examples & snippets

- Create deterministic temp root in tests:
- var root = Path.Combine(Path.GetTempPath(), "GisConverter.Tests", Guid.NewGuid().ToString("N")); Directory.CreateDirectory(root);
- Create a small zip with an entry:
- using (var fs = File.Create(zipPath)) using (var za = new ZipArchive(fs, ZipArchiveMode.Create)) { var ze = za.CreateEntry("doc.kml"); using var s = ze.Open(); using var sw = new StreamWriter(s, Encoding.UTF8); sw.Write("<kml/>"); }

- Example conversion assertion (integration):
- var maybeExt = FileExtensionHelpers.FromOption("Shapefile"); var dot = FileExtensionHelpers.ToDotExtension(maybeExt ?? FileExtension.Shapefile); var files = Directory.GetFiles(outDir, "*" + dot, SearchOption.AllDirectories); Assert.NotEmpty(files);
- 
---

## Contact & reporting
When filing issues about failing converter tests include:
- The failing test name and full stack trace.
- Captured stdout/stderr and `TestLogger` output for the run.
- Any sample artifact used by the test (or its expected location under `TestsApp/TestData`).