Location Validation

Every asset uploaded through POST /ingestions/assets is automatically validated and normalized before metric computation begins. This page describes the full validation pipeline: data normalization, plot-level geometric and risk checks, and facility-level operational plausibility scoring.


Data Normalization

Supported File Formats

The ingestion pipeline accepts the following geospatial and tabular formats:

CategoryFormats
Vector geospatialGeoJSON, Shapefile (.shp + sidecar files), GeoPackage (.gpkg), GeoParquet, KML, KMZ
TabularCSV, XLSX (Excel)
UnstructuredPDF, TXT

AI-Powered Schema Detection

Uploaded datasets are processed by an AI-powered normalization engine (Gemini) that automatically detects the file schema regardless of column naming conventions. This handles:

  • Column auto-detection: The system identifies columns containing coordinates, plot identifiers, supplier names, addresses, commodity types, and other relevant fields. Common aliases (e.g., lat, latitude, y_coord, LAT, Supplier Name, operator) are recognized across languages and naming conventions.
  • Column mapping: Detected columns are mapped to the standard internal schema: supplier name, address, location geometry, commodity, production volume, and other EUDR-relevant fields.
  • Commodity detection: Commodities are identified using a combination of AI classification and heuristic fallbacks. When a commodity column is present, values are normalized to the supported set. When absent, the system infers the commodity from context (file name, other column values, geographic location).

Coordinate Parsing and Projection

Coordinates provided in various formats are parsed and standardized to WGS84 decimal degrees:

Input FormatExampleHandling
Decimal degrees-3.4567, 28.1234Direct use
Degrees-minutes-seconds (DMS)3 27'24"S, 28 07'24"EParsed and converted
UTM coordinatesZone 36S: 456789, 9876543Zone detected, reprojected to WGS84
Swapped lat/lon28.1234, -3.4567 (lon, lat order)Detected and corrected

Geocoding

When uploaded records contain addresses or place names but no geometry, the system geocodes them to coordinates. This is common for facility records submitted as spreadsheets with address columns rather than coordinate columns.

Additional Cleaning

  • Duplicate detection: Duplicate or near-duplicate records are identified and flagged.
  • Character encoding normalization: Unicode normalization for location names, operator names, and other text fields.
  • EUDR payload generation: Normalized data is structured into the format required for downstream EUDR compliance processing.

Plot Validation

Plot validation uses a pillar-based risk structure. Each uploaded plot is evaluated across multiple independent risk pillars, and the results are combined into an overall risk assessment.

Risk Pillars

1. Plot Validity

Assesses geometric quality, commodity presence, and protected area status. This pillar runs 15 geometric checks on every plot polygon, each producing a weighted contribution to the shape validity score.

Geometric checks:

CheckDescription
Area validationPlot area must fall within commodity-specific bounds. Extremely small plots (below detection algorithm resolution) or implausibly large plots are flagged.
Self-intersectionThe polygon must not self-intersect. Self-intersecting geometries produce ambiguous area calculations and are automatically repaired where possible (buffer-zero correction).
Duplicate verticesConsecutive identical or near-identical vertices are detected.
Angle validation (spike detection)Interior angles are checked for extremely acute angles (near-zero or near-360 degrees) that indicate digitization errors or slivers.
Distance validationMaximum distance between vertices is checked to detect implausibly elongated shapes.
Precision validationCoordinates with insufficient decimal places (e.g., integer degrees) are flagged. Minimum precision of approximately 10 m is required.
Boundary segment lengthShort boundary segments that suggest over-digitization or artifacts are detected.
Overlap detectionPlots within the same collection are checked for mutual overlap. Significant overlap may indicate duplicate submissions or boundary errors.
Nearest neighbor distanceDistance to the closest other plot in the collection is measured. Extremely close but non-overlapping plots may indicate boundary drawing errors.
Geometry collection processingNested geometry collections are flattened and simplified where possible.
Multipolygon flatteningMultipolygon geometries are decomposed into individual polygons for per-plot analysis.
Triangular geometryTriangular shapes are flagged as potentially artificial or placeholder geometries.
Commodity presenceThe plot location is checked against commodity probability maps to verify the declared commodity is plausibly grown in that area.
Protected area overlap (WDPA)Plot boundaries are checked against the World Database on Protected Areas. Overlap with protected areas triggers a legality risk flag.

Shape validity issue weights:

Each geometric issue contributes to the overall shape validity score with the following weights:

IssueWeight
Intersects another polygon0.30
Self-intersection0.25
Commodity absence0.25
Distance to nearest neighbor0.20
Protected area overlap0.20
Area too small0.10
Maximum distance0.10
Maximum angle0.10
Bad precision0.05
Short boundary segments0.05
Triangular geometry0.05
Duplicate vertices0.05
Nested geometry collection0.05
Simplifiable collections0.05

2. Deforestation

Plot-level deforestation detection using multiple alert systems:

  • GLAD alerts (Global Land Analysis and Discovery)
  • TMF alerts (Tropical Moist Forests)
  • Hansen Global Forest Change

Deforestation analysis is restricted to natural forest. Managed forest areas are exempted using the following rules:

  • GFC (Global Forest Change) subtype 20 areas are excluded (plantation forest)
  • Nature Trace Forest Typology (ForTy) planted and plantation forest classes are excluded globally
  • Country-level managed forest exemptions apply for Canada and the United States

Only post-EUDR-cutoff deforestation is considered for compliance purposes.

3. Cherry-Picking

Detects geographic selection bias: whether plots have been drawn to avoid known deforestation hotspots. The system analyzes the vicinity within the collection's extent bounding box and compares deforestation rates inside plots versus the surrounding area.

Risk LevelVicinity Deforestation DiscrepancyInterpretation
Low<= 5%No evidence of selective boundary drawing
Medium<= 20%Some indication of avoidance patterns; warrants review
High> 20%Strong evidence that plots were drawn to exclude deforestation

4. Circumvention

Production volume plausibility check. When a production_volume parameter is provided, the system compares the declared volume against yield estimates for the plot area and commodity. This detects volume overstatement -- declaring more production than the land area could plausibly support, which may indicate that production from undisclosed (potentially deforested) land is being attributed to the declared plots.

5. Legality

AI-powered web intelligence assessment using Gemini with Google Search grounding. This is a collection-level check (not per-plot) that searches for:

  • Post-2021 deforestation incidents linked to the operator or region
  • Land rights disputes or contested tenure
  • Environmental violation records
  • Sanctions or enforcement actions
  • Media reports of illegal activity

6. Overall

A weighted combination of all pillar scores into a single risk assessment. The overall result includes an investigate list identifying the specific issues that require attention.


Facility Validation

Facilities (mills, silos, warehouses, processing plants) are validated for operational plausibility and environmental risk. Each facility receives a confidence score from 0 to 1 based on multiple independent indicators.

Supported Commodities

CommodityDedicated Probability Maps
RubberYes
Palm oilYes
CocoaYes
CoffeeYes
SoyYes
TimberYes
CattleYes
Other / unsupportedPlantation forest mask fallback

For commodities without dedicated probability maps, the system falls back to general plantation forest masks for commodity presence scoring.

Validation Checks

Commodity Presence

The facility location is checked against commodity probability maps within a configurable radius (default: 5000 m). A facility declaring palm oil in an area with no palm cultivation signal is flagged. The radius accounts for the fact that facilities may be located in towns or industrial zones adjacent to, rather than within, production areas.

Infrastructure Access

Connectivity to transportation infrastructure is assessed using Overture Maps and Google Places data:

  • Road access: Distance to the nearest road network segment
  • Waterway access: Proximity to navigable waterways (relevant for commodities like palm oil where river transport is common)
  • Port access: Distance to the nearest port facility

Facilities with no road access within a reasonable distance are flagged as potentially mislocated.

Building and Industrial Presence

High-resolution building footprint and industrial facility data from Overture Places is used to verify that structures exist at or near the declared location, within a 100 m radius. Scoring is graduated by facility size:

Detected FeatureScore Contribution
Large industrial building+0.23
Medium building+0.12
No buildings detectedNo contribution (lowers confidence)

An empty location with no built-up area is flagged as potentially non-existent.

Business Context

Nearby points of interest (POIs) from Overture Maps and other sources are queried for industrial, agricultural, and commercial features. Matching POI types (e.g., "silo", "mill", "warehouse", "processing plant") increase confidence that the location is a real operational facility.

Geographic Suitability

Latitude constraints are applied per commodity. For example, palm oil facilities at latitudes above 25 N or below 25 S are flagged as geographically implausible.

Deforestation Check

The facility's surrounding area is screened for deforestation signals:

  • Resolution: 30 m
  • Radius: 50 km
  • Forest type: Natural forest only (managed forest excluded)
  • Time period: Post-EUDR cutoff date only

Facilities surrounded by high deforestation activity are flagged for enhanced due diligence.

Legality Risk

AI-powered web intelligence assessment (same Gemini + Google Search grounding engine as plot-level legality checks). Searches for regulatory violations, sanctions, and environmental incidents associated with the facility operator and location.

Confidence Scoring

Each facility receives a composite confidence score:

Score RangeInterpretation
0.8 -- 1.0High confidence. Multiple indicators confirm facility presence and commodity alignment.
0.5 -- 0.8Moderate confidence. Some indicators are positive, but others are missing or inconclusive. Manual verification recommended.
0.2 -- 0.5Low confidence. Few indicators support the declared facility. Likely mislocated or misclassified.
0.0 -- 0.2Very low confidence. No supporting indicators found. The location may be incorrect or the facility may not exist.

Facilities with confidence scores below 0.5 are flagged for manual review before supply shed analysis proceeds.