· بقلم Cayetano Benavent

Portolan v0.3Meeting geospatial data where it already lives

كل المقالات

هذه المقالة متاحة بالإنجليزية فقط. بقية الموقع مترجمة.

Portolan v0.3.0 came out on November 17th. When we started on it, most of the open questions came from the same place: people who wanted to publish data that didn't quite fit the specification yet. Tables written by lakehouse engines. Raster archives that a browser couldn't show at a glance. Data cubes in Zarr. Catalogs so large that finding a collection meant reading hundreds of JSON files. This release is our answer to those questions. Some of the answers are firm, and some are first drafts that we want you to try.

You'll see two labels in this post. Normative changes are part of conformance. Validators check them, and a conformant catalog follows them. Incubating specs are written down, with their own requirement IDs, but they aren't part of conformance yet. They can change before we make them normative, and your feedback decides how. The full changelog lists every change.

Parquet with native geometry types (normative)

Until v0.3 of the spec, Portolan vector data had to be GeoParquet. Meanwhile, Parquet added its own support for geometry. The format now has native GEOMETRY and GEOGRAPHY types, and more engines write them, including Apache Iceberg writers. A table written that way is clearly vector data, but earlier validators treated it as a plain table, because it had no GeoParquet geo metadata.

v0.3 accepts GeoParquet 1.1, GeoParquet 2.0, and Parquet with native geometry types, and it applies the same vector rules to all three. We recommend GeoParquet 2.0 when your tools can write it. Geometry uses Parquet's own type, so any Parquet reader recognizes it, and every row group has bounding box statistics. The geo metadata is still there too, and it records the CRS in the file.

For a data team, this means files that come out of the lakehouse they already use can go straight into a catalog.

Better raster visualization (normative)

With a collection of hundreds of raster tiles, a browser either draws one tile or tries to draw all of them. v0.3 asks large raster collections to include an overview COG: one small image in a web map projection that shows the whole collection at once.

Classified rasters, such as land cover, now describe their classes with the STAC classification extension. A client can then draw a legend that says "forest" and "water" and not "class 3" and "class 7". Imagery should also declare its nodata value, so clients don't draw the empty edges of a scene as black.

Improved metadata and docs (normative)

Portolan has always asked for documentation next to the data, but v0.3 takes it a step further. A column called dtvw means nothing to people or agents unless the catalog tells you it's the average weekday traffic counts. Going forward, table columns should have descriptions, and assets should have a title and a description.

Data in the bucket where it lives (normative)

Many catalogs keep their data in S3 or Google Cloud Storage. v0.3 lets assets link with s3:// and similar schemes, as long as they also give an https alternate for clients that only speak HTTP. Servers that are S3-compatible but not on AWS are described with the STAC storage extension so that clients know where to send the request. Because range requests are now required only for the data files that need them (COG, Parquet, PMTiles, and COPC), a host that serves only the metadata (like a metadata-only GitHub repo) can be a plain static server.

This is also where v0.3 introduces breaking changes:

  • Partition globs must be absolute bucket URLs, such as s3:// or gs://. A reader has to list the files that match a glob, and you can't list files over plain HTTP or with a relative path. In October, this affected 9 of 37 collections in the registry. The fix is to replace the relative glob with the bucket URL.
  • Each data asset links to exactly one file.
  • PMTiles can't be a collection's only data. It's the visual layer, so it has to point to the GeoParquet it was made from.

New metrics for spatial ordering

Rows in a GeoParquet file must still follow a spatial order. That rule is still normative, but we've changed how we check it. The old test looked at how much the row groups overlapped and failed files that were well sorted but had only a few row groups. We've replaced this with an incubating method that tries to more intelligently measure what spatial order is a proxy for: the share of a file that a typical query can skip. Until we've calibrated these metrics on real catalogs, rashid will report these measures as warnings only.

STAC-GeoParquet as a catalog (incubating)

v0.3 defines how a Portolan catalog publishes itself as STAC-GeoParquet . In short, the whole catalog becomes one table you can query. Each catalog and collection in the tree gets a row, with its extent, time range, keywords, links, and the full original JSON. The JSON is still the normative form. The layout follows the column layout proposed upstream in the stac-geoparquet spec , which is still under discussion and not released. The schema can still change a bit; Portolan will follow upstream when it does. Many catalogs in the registry already publish one, and the registry will publish one for all of them. Keep an eye out for our upcoming post about STAC-GeoParquet as a catalog.

Zarr (incubating)

Portolan started with GeoParquet , PMTiles , and COG . But a lot of climate, weather, and Earth observation data is published as Zarr , which is a legitimately cloud-native file format, so v0.3 adds an incubating Zarr profile for Zarr v3 raster stores. If you publish data cubes, this is the part of the release where we most need your feedback; feel free to add your comments on this tracking issue on GitHub .

Apache Iceberg (incubating)

Iceberg is new in v0.3. The incubating convention lets a catalog publish an Apache Iceberg table over the GeoParquet it already serves. STAC is still the catalog, and describes the data. The Iceberg table is a second way in for the engines that speak Iceberg.

Both paths end at the same bytes. The live data files of the table are exactly the collection's data assets. The table adds metadata but no second copy of the data, nor does a reader need a server. The collection describes the table with the STAC Iceberg extension and points to the table's metadata file so that an engine can open the table from that file alone.

This matters most for static catalogs. Because the table's manifests list every file, a client can read a partitioned collection over plain HTTPS without listing the bucket. Per-file bounds let an engine skip a file without opening it. The convention uses Iceberg format version 3, the first with a native geometry type, over GeoParquet 2.0 data files. It works in both directions: a publisher can register existing GeoParquet as a table, or point the catalog at files an Iceberg engine wrote first.

Validate with rashid 0.4

The canonical spec validator, rashid, has also been bumped v0.4.0 which implements the spec's v0.3 rules (spec and validator versioning are not aligned). rashid bundles the v0.3.0 profile schema and checks the new rules, such as native geospatial Parquet, column descriptions, raster classes, and absolute partition globs. For the incubating parts, it checks a Zarr store against the incubating profile and reports spatial order as a warning.

rashid applies the newest rules it knows, whatever version your catalog declares, and it reports both versions. So it shows you what v0.3 would flag before you move.

uv tool install rashid
rashid check path/to/catalog

Try it

Read the specification , and declare v0.3 in your catalog with https://schemas.portolan-sdi.org/portolan/v0.3.0/schema.json in stac_extensions. The earlier schemas are still published, so nothing changes for a catalog until you move it. Then run rashid, and tell us what works and what doesn't in the Portolan Google Group or the Portolan channel in Cloud-Native Geo Slack.

Thanks to everyone in the community who contributed to this release with code, reviews, issues, and discussion.