Location Validation
Every asset uploaded through POST /ingestions/assets is automatically validated and normalized before metric computation begins. This page describes the full validation pipeline: data normalization, plot-level geometric and risk checks, and facility-level operational plausibility scoring.
▶Data Normalization
▶Supported File Formats
The ingestion pipeline accepts the following geospatial and tabular formats:
| Category | Formats |
|---|---|
| Vector geospatial | GeoJSON, Shapefile (.shp + sidecar files), GeoPackage (.gpkg), GeoParquet, KML, KMZ |
| Tabular | CSV, XLSX (Excel) |
| Unstructured | PDF, TXT |
▶AI-Powered Schema Detection
Uploaded datasets are processed by an AI-powered normalization engine (Gemini) that automatically detects the file schema regardless of column naming conventions. This handles:
- Column auto-detection: The system identifies columns containing coordinates, plot identifiers, supplier names, addresses, commodity types, and other relevant fields. Common aliases (e.g.,
lat,latitude,y_coord,LAT,Supplier Name,operator) are recognized across languages and naming conventions. - Column mapping: Detected columns are mapped to the standard internal schema: supplier name, address, location geometry, commodity, production volume, and other EUDR-relevant fields.
- Commodity detection: Commodities are identified using a combination of AI classification and heuristic fallbacks. When a commodity column is present, values are normalized to the supported set. When absent, the system infers the commodity from context (file name, other column values, geographic location).
▶Coordinate Parsing and Projection
Coordinates provided in various formats are parsed and standardized to WGS84 decimal degrees:
| Input Format | Example | Handling |
|---|---|---|
| Decimal degrees | -3.4567, 28.1234 | Direct use |
| Degrees-minutes-seconds (DMS) | 3 27'24"S, 28 07'24"E | Parsed and converted |
| UTM coordinates | Zone 36S: 456789, 9876543 | Zone detected, reprojected to WGS84 |
| Swapped lat/lon | 28.1234, -3.4567 (lon, lat order) | Detected and corrected |
▶Geocoding
When uploaded records contain addresses or place names but no geometry, the system geocodes them to coordinates. This is common for facility records submitted as spreadsheets with address columns rather than coordinate columns.
▶Additional Cleaning
- Duplicate detection: Duplicate or near-duplicate records are identified and flagged.
- Character encoding normalization: Unicode normalization for location names, operator names, and other text fields.
- EUDR payload generation: Normalized data is structured into the format required for downstream EUDR compliance processing.
▶Plot Validation
Plot validation uses a pillar-based risk structure. Each uploaded plot is evaluated across multiple independent risk pillars, and the results are combined into an overall risk assessment.
▶Risk Pillars
▶1. Plot Validity
Assesses geometric quality, commodity presence, and protected area status. This pillar runs 15 geometric checks on every plot polygon, each producing a weighted contribution to the shape validity score.
Geometric checks:
| Check | Description |
|---|---|
| Area validation | Plot area must fall within commodity-specific bounds. Extremely small plots (below detection algorithm resolution) or implausibly large plots are flagged. |
| Self-intersection | The polygon must not self-intersect. Self-intersecting geometries produce ambiguous area calculations and are automatically repaired where possible (buffer-zero correction). |
| Duplicate vertices | Consecutive identical or near-identical vertices are detected. |
| Angle validation (spike detection) | Interior angles are checked for extremely acute angles (near-zero or near-360 degrees) that indicate digitization errors or slivers. |
| Distance validation | Maximum distance between vertices is checked to detect implausibly elongated shapes. |
| Precision validation | Coordinates with insufficient decimal places (e.g., integer degrees) are flagged. Minimum precision of approximately 10 m is required. |
| Boundary segment length | Short boundary segments that suggest over-digitization or artifacts are detected. |
| Overlap detection | Plots within the same collection are checked for mutual overlap. Significant overlap may indicate duplicate submissions or boundary errors. |
| Nearest neighbor distance | Distance to the closest other plot in the collection is measured. Extremely close but non-overlapping plots may indicate boundary drawing errors. |
| Geometry collection processing | Nested geometry collections are flattened and simplified where possible. |
| Multipolygon flattening | Multipolygon geometries are decomposed into individual polygons for per-plot analysis. |
| Triangular geometry | Triangular shapes are flagged as potentially artificial or placeholder geometries. |
| Commodity presence | The plot location is checked against commodity probability maps to verify the declared commodity is plausibly grown in that area. |
| Protected area overlap (WDPA) | Plot boundaries are checked against the World Database on Protected Areas. Overlap with protected areas triggers a legality risk flag. |
Shape validity issue weights:
Each geometric issue contributes to the overall shape validity score with the following weights:
| Issue | Weight |
|---|---|
| Intersects another polygon | 0.30 |
| Self-intersection | 0.25 |
| Commodity absence | 0.25 |
| Distance to nearest neighbor | 0.20 |
| Protected area overlap | 0.20 |
| Area too small | 0.10 |
| Maximum distance | 0.10 |
| Maximum angle | 0.10 |
| Bad precision | 0.05 |
| Short boundary segments | 0.05 |
| Triangular geometry | 0.05 |
| Duplicate vertices | 0.05 |
| Nested geometry collection | 0.05 |
| Simplifiable collections | 0.05 |
▶2. Deforestation
Plot-level deforestation detection using multiple alert systems:
- GLAD alerts (Global Land Analysis and Discovery)
- TMF alerts (Tropical Moist Forests)
- Hansen Global Forest Change
Deforestation analysis is restricted to natural forest. Managed forest areas are exempted using the following rules:
- GFC (Global Forest Change) subtype 20 areas are excluded (plantation forest)
- Nature Trace Forest Typology (ForTy) planted and plantation forest classes are excluded globally
- Country-level managed forest exemptions apply for Canada and the United States
Only post-EUDR-cutoff deforestation is considered for compliance purposes.
▶3. Cherry-Picking
Detects geographic selection bias: whether plots have been drawn to avoid known deforestation hotspots. The system analyzes the vicinity within the collection's extent bounding box and compares deforestation rates inside plots versus the surrounding area.
| Risk Level | Vicinity Deforestation Discrepancy | Interpretation |
|---|---|---|
| Low | <= 5% | No evidence of selective boundary drawing |
| Medium | <= 20% | Some indication of avoidance patterns; warrants review |
| High | > 20% | Strong evidence that plots were drawn to exclude deforestation |
▶4. Circumvention
Production volume plausibility check. When a production_volume parameter is provided, the system compares the declared volume against yield estimates for the plot area and commodity. This detects volume overstatement -- declaring more production than the land area could plausibly support, which may indicate that production from undisclosed (potentially deforested) land is being attributed to the declared plots.
▶5. Legality
AI-powered web intelligence assessment using Gemini with Google Search grounding. This is a collection-level check (not per-plot) that searches for:
- Post-2021 deforestation incidents linked to the operator or region
- Land rights disputes or contested tenure
- Environmental violation records
- Sanctions or enforcement actions
- Media reports of illegal activity
▶6. Overall
A weighted combination of all pillar scores into a single risk assessment. The overall result includes an investigate list identifying the specific issues that require attention.
▶Facility Validation
Facilities (mills, silos, warehouses, processing plants) are validated for operational plausibility and environmental risk. Each facility receives a confidence score from 0 to 1 based on multiple independent indicators.
▶Supported Commodities
| Commodity | Dedicated Probability Maps |
|---|---|
| Rubber | Yes |
| Palm oil | Yes |
| Cocoa | Yes |
| Coffee | Yes |
| Soy | Yes |
| Timber | Yes |
| Cattle | Yes |
| Other / unsupported | Plantation forest mask fallback |
For commodities without dedicated probability maps, the system falls back to general plantation forest masks for commodity presence scoring.
▶Validation Checks
▶Commodity Presence
The facility location is checked against commodity probability maps within a configurable radius (default: 5000 m). A facility declaring palm oil in an area with no palm cultivation signal is flagged. The radius accounts for the fact that facilities may be located in towns or industrial zones adjacent to, rather than within, production areas.
▶Infrastructure Access
Connectivity to transportation infrastructure is assessed using Overture Maps and Google Places data:
- Road access: Distance to the nearest road network segment
- Waterway access: Proximity to navigable waterways (relevant for commodities like palm oil where river transport is common)
- Port access: Distance to the nearest port facility
Facilities with no road access within a reasonable distance are flagged as potentially mislocated.
▶Building and Industrial Presence
High-resolution building footprint and industrial facility data from Overture Places is used to verify that structures exist at or near the declared location, within a 100 m radius. Scoring is graduated by facility size:
| Detected Feature | Score Contribution |
|---|---|
| Large industrial building | +0.23 |
| Medium building | +0.12 |
| No buildings detected | No contribution (lowers confidence) |
An empty location with no built-up area is flagged as potentially non-existent.
▶Business Context
Nearby points of interest (POIs) from Overture Maps and other sources are queried for industrial, agricultural, and commercial features. Matching POI types (e.g., "silo", "mill", "warehouse", "processing plant") increase confidence that the location is a real operational facility.
▶Geographic Suitability
Latitude constraints are applied per commodity. For example, palm oil facilities at latitudes above 25 N or below 25 S are flagged as geographically implausible.
▶Deforestation Check
The facility's surrounding area is screened for deforestation signals:
- Resolution: 30 m
- Radius: 50 km
- Forest type: Natural forest only (managed forest excluded)
- Time period: Post-EUDR cutoff date only
Facilities surrounded by high deforestation activity are flagged for enhanced due diligence.
▶Legality Risk
AI-powered web intelligence assessment (same Gemini + Google Search grounding engine as plot-level legality checks). Searches for regulatory violations, sanctions, and environmental incidents associated with the facility operator and location.
▶Confidence Scoring
Each facility receives a composite confidence score:
| Score Range | Interpretation |
|---|---|
| 0.8 -- 1.0 | High confidence. Multiple indicators confirm facility presence and commodity alignment. |
| 0.5 -- 0.8 | Moderate confidence. Some indicators are positive, but others are missing or inconclusive. Manual verification recommended. |
| 0.2 -- 0.5 | Low confidence. Few indicators support the declared facility. Likely mislocated or misclassified. |
| 0.0 -- 0.2 | Very low confidence. No supporting indicators found. The location may be incorrect or the facility may not exist. |
Facilities with confidence scores below 0.5 are flagged for manual review before supply shed analysis proceeds.