What the Open Data Directive asks of a high-value dataset portal
An EU public sector body that must publish high-value datasets under the Open Data Directive (EU) 2019/1024 needs to make datasets in six categories available free of charge, in machine-readable form, through APIs and, where relevant, as bulk download. The six categories are geospatial, earth observation and environment, meteorological, statistics, companies and company ownership, and mobility. Commission Implementing Regulation (EU) 2023/138 lists the specific datasets held by public sector bodies and requires them to be available for re-use under CC0 or CC BY 4.0, or an equivalent or less restrictive open licence. The Directive entered into force on 16 July 2019 with a transposition deadline of 16 July 2021, and the implementing regulation applies from 9 June 2024. A CKAN portal can cover free access, machine-readable resources, a DataStore Data API for filtered and paginated queries, bulk download of resource files, and a licence field set per dataset. Confirm with your national transposition law whether any additional duties apply to your body.
The software covers the technical side, but the law is met by decisions as much as by configuration. Before launch, the team should map each dataset in the 2023/138 list to an owner, a licence, a publication date and a format, then check that the licence recorded in CKAN matches the one the body has authority to grant. Charges for re-use, exclusive arrangements and restrictions on onward publication are the usual points where a portal fails the Directive even when the software works. Ask your legal team to confirm which datasets fall under exceptions, such as those held by bodies that generate revenue to cover costs, and record the answer in the dataset metadata so the decision can be audited later.
Operational duties also recur after go-live. Updates to a dataset should reach the portal on the schedule the owning body commits to, and broken API links or stale resources count as compliance failures in practice. The team should decide who monitors the DataStore loads, who responds when an XLoader job fails on a new file version, and how changes to licences or datasets are logged. Plan these roles and runbooks before the first publication, because the regulation applies to the data as it is kept over time, not only on the day it is first released.
Mapping each Directive requirement to portal configuration
This table maps each Open Data Directive requirement to what the portal must do and the CKAN component that handles it. Read each row as a question to answer for your own datasets, since the Commission's dataset list and each dataset's licence decide which rows apply in practice. The main takeaway is that most obligations are met by configuration and process, but the list of datasets and their licences must be confirmed by your organization before go-live.
| Requirement | What the portal must do | How to configure it |
|---|---|---|
| Free of charge | No fee, registration gate or payment step to download high-value datasets | Keep resources public in CKAN, avoid paywall plugins on these datasets, and document the no-charge position in the dataset description |
| Machine-readable format | Publish in formats software can process, such as CSV, JSON or GeoJSON, not scanned images or PDF tables | Set resource formats in CKAN and reject uploads that are only PDF for high-value datasets during the publishing workflow |
| Access through APIs | Let users query the data programmatically without manual downloads | Enable the DataStore for tabular resources and expose the Data API for filtering, search and pagination |
| Bulk download | Offer complete dataset files where relevant, not only query results | Provide the original file as a resource and, where needed, a full DataStore export for each dataset |
| Loading tabular files | Make uploaded or linked files queryable through the API | Use XLoader, the officially supported loader, rather than the legacy DataPusher |
| Open licence | Each high-value dataset carries CC0 or CC BY 4.0, or an equivalent or less restrictive licence | Set the licence field per dataset in CKAN and confirm each licence with the data owner before publication |
| Commission dataset list | Only the datasets named in the Implementing Regulation fall under the high-value rules | Tag and document in-scope datasets, and confirm the list with your legal team against the Implementing Regulation (EU) 2023/138 |
| Metadata exchange | Describe datasets in a form that European catalogs can read | Use ckanext-dcat with the DCAT-AP profile your catalog targets, and validate the RDF output |
Which open-source platforms fit high-value dataset publication
High-value dataset publication is a catalog and data access function first, so the choice comes down to which platform manages metadata, serves data through an API and bulk download, and exports standard metadata. The cards below cover the primary fit, the versions and licence terms to confirm, and where general content tools fit around the catalog.
CKAN as the primary fit
CKAN is the platform built for this job: a data catalog with dataset records, resources, organizations and a DataStore that serves tabular data through a Data API. It is licensed under AGPL-3.0, so modifications you distribute or run as a network service carry source obligations that your legal team should review before you customize it. Confirm that your dataset categories map to CKAN's dataset and resource model before you commit to it.
CKAN release policy and support
CKAN targets about two minor releases a year, and the tech team supports the latest patch of the current minor version plus the previous minor version. Patch releases exclude schema changes, while minor releases may break compatibility, so plan upgrades as projects, not routine patches. Budget for an upgrade path in your support terms, and confirm which minor version your implementation partner will support after go-live.
Runtime stack and supported versions
A source install needs Python 3.10 or newer, PostgreSQL 12 or newer, Solr 9 (recommended) or Solr 8, and Redis, with uWSGI and Nginx recommended for production. Solr handles search, so size it for your dataset count and query load. Confirm which operating systems your hosting team supports, because the package install guide covers Ubuntu 20.04 and 22.04 while other distributions use source installs.
DataStore and XLoader for API access
The DataStore keeps structured resource data in a separate PostgreSQL database with a write user and a read-only user, which supports the API and bulk access expected for high-value datasets. XLoader is the supported loader, and DataPusher is legacy, so any new build should start with XLoader. Test loads against your largest files before launch, because load time and file size limits depend on your configuration.
Headless CMS and commerce platforms do not fit
Headless CMS tools such as Strapi, Payload and Directus manage content models and APIs, but they do not provide dataset catalog records, harvesting, DataStore-style tabular APIs or DCAT metadata export out of the box. Commerce platforms like Magento, WooCommerce or Saleor solve cart and order problems that do not exist in a public data catalog. Choose them only if a separate team needs them for an unrelated site, and avoid forcing either into the catalog role.
General CMS for surrounding pages
A general content management system such as Drupal or WordPress can host the public-facing pages around the catalog, including the portal home page, guidance on licences, news and accessibility statements. Keep the catalog itself on CKAN and link the two through a shared theme or reverse proxy, so dataset URLs stay stable. Confirm the content editor roles and accessibility obligations for the CMS before you split responsibilities.
Evidence a reviewer or auditor will ask for
What dataset inventory proves a public body covers the high-value categories?
An inventory that maps every dataset to one of the six high-value categories and to the specific datasets listed in Commission Implementing Regulation (EU) 2023/138 is the first item a reviewer asks for. In CKAN, each dataset is a package with its own metadata record, so the inventory is produced by exporting packages with their organization, tags and a custom field or extra that records the high-value category. Keep the export dated and versioned so you can show what was held when the regulation applied from 9 June 2024. Confirm with your national oversight body whether it expects a specific template, a list of datasets not yet published, or a reasoned exemption for datasets outside scope.
How do I show every high-value resource carries an open licence?
Licence records must show, per resource, a licence that is CC0 or CC BY 4.0, or an equivalent or less restrictive open licence, as the implementing regulation requires. CKAN stores a licence identifier on each dataset, so the evidence is a report that checks the licence field across all high-value packages and flags any dataset that is missing one or uses a different licence. Where a resource carries its own terms, keep those terms in the resource metadata and in the publishing record. Confirm with your legal team which licence identifiers your portal accepts and how exceptions are documented.
What API and bulk download tests should we keep as evidence?
Keep dated test results showing that each high-value resource is reachable through an API and, where relevant, as a bulk download, and that the files are machine readable. For tabular resources loaded into the DataStore, the CKAN Data API should return records for filter, search, pagination and SQL queries, and the bulk file should match the source. Use XLoader for loading, since it is the officially supported tool and DataPusher is legacy. Script the checks against the production URLs, store the output with the date and the CKAN version, and repeat them after each upgrade. Confirm which download formats your oversight body accepts as machine readable.
Which DCAT-AP output validation records should we store?
Store the output of a validation run against the DCAT-AP profile your country uses, together with the RDF files it checked. The ckanext-dcat extension provides DCAT-AP v1, v2 and v3 schemas and RDF endpoints, so the evidence is the serialized catalog for each version in use, the validator report and the version of the extension. Record any harvested records from other catalogs separately, since they carry their own quality issues. Confirm with your national metadata authority which DCAT-AP version and national profile it requires, because the requirement differs between member states.
What release and patch records does CKAN need to show?
A reviewer will expect a record of the CKAN version in production, the patch level, and the dates of each upgrade and security fix. CKAN targets about two minor releases a year, supports the latest patch of the current minor and the previous minor, and drops outdated patch releases once a newer patch is out. Keep a change log per upgrade with the version installed, the release notes read, the test results after the upgrade, and the rollback plan. CKAN 2.12.0 was released on August 26, 2026, with maintenance releases 2.11.6 and 2.10.11 on the same day. Patch releases exclude schema changes, but minor releases may include backwards-incompatible changes, so the upgrade record must show which type was applied.
Which access logs prove who read the DataStore and when?
Access evidence covers the separate PostgreSQL database that holds DataStore tables, which uses a write user and a read-only user. Logs should show that the public Data API connects through the read-only role, that the write role is used only by the loading process, and that connections and failed attempts are retained for the period your oversight body sets. This evidence comes from the PostgreSQL logging configuration, the web server access logs for the Data API endpoints, and the CKAN activity stream for dataset changes. Confirm the retention period, the log location and who may review the logs with your security and records team before you agree on the monitoring design.
Who should confirm the evidence pack meets our oversight body's requirements?
Your national open data authority, the body that supervises your public sector, and your internal audit or records function set what counts as sufficient evidence, so they should confirm the list before you collect it. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications, and independent auditors or certification bodies make those decisions. In a fixed-scope engagement, the written scope should name each evidence item, the CKAN component that produces it, the format, the owner and the retention period. Ask the oversight body whether it accepts exported reports, screenshots or only signed records.
What a CKAN installation alone does not satisfy
Installing CKAN gives a public body a catalog, a DataStore and an API, but it does not decide which datasets are high-value, who licenses them, or how they stay current. The cards below cover the work and evidence that sit around the platform, and they start from one point: software does not decide which datasets fall within the six high-value categories.
Inventory and high-value dataset selection
Commission Implementing Regulation (EU) 2023/138 lists the specific high-value datasets that public sector bodies must make available, so the first task is mapping what your body holds against that list. Software cannot make that determination. Check which datasets fall within the six categories, which are already published elsewhere, and which hold data that cannot be shared at all. Record the result in an inventory that the portal's metadata can reference.
Licence decisions by data owners
Each dataset must be available for re-use under CC0 or CC BY 4.0, or an equivalent or less restrictive open licence. Those decisions belong to the data owners in each department, not to the portal team. Confirm who holds authority to license each dataset, document the licence in the metadata, and check whether any third-party rights or personal data in the source files block an open licence before the dataset goes live.
Legal review of re-use terms
Re-use conditions, disclaimers and attribution wording should be reviewed by counsel before publication, especially where datasets combine public and third-party sources. The portal can display the terms, but it cannot judge whether they are lawful or sufficient. Ask counsel to confirm the national transposition rules, the body's own statutory basis, and how corrections or withdrawals of a dataset will be handled and logged.
Publishing workflows and roles
CKAN supports organizations and user roles, but the approval chain from data owner to publisher to reviewer is a process you design and document. Define who can create, edit, approve and withdraw a dataset, and who answers requests for re-use. Map those roles to your identity provider so that access changes follow staff movements. Without this, datasets are published by whoever has access, which is hard to defend in an assessment.
Update automation and DataStore loading
High-value datasets often change on a schedule, and stale files undermine re-use. Plan how each resource is refreshed, how XLoader or any other loader is triggered after an update, and how failed loads are detected and reported. Confirm that the DataStore read-only user is the only credential used by the public API, and that write credentials stay within your publishing pipeline.
Patch and minor release monitoring
CKAN supports the latest patch of the current minor version and the previous minor version, and an outdated patch loses support once a newer one ships. CKAN 2.12.0 was released on August 26, 2026, with maintenance releases 2.11.6 and 2.10.11 the same day. Assign an owner to track each release, test extensions such as ckanext-dcat against it, and schedule upgrades, because minor releases can include backwards-incompatible changes.
Security controls for NIS2 or national frameworks
NIS2 covers central government entities regardless of size and lists public administration as a sector of high criticality, and Member States may extend it to local bodies. If your body falls under it or a national security framework, you must evidence controls such as access management, logging, incident handling and supplier security. CKAN supplies some of these capabilities, but the evidence, the risk assessment and the operating procedures belong to your organization.
How the build proceeds from inventory to first publication
The sequence moves from an agreed dataset inventory to a controlled first publication of high-value datasets on a self-hosted CKAN portal. Each phase closes with a written check, so the public body can confirm the portal meets the Directive's open access and machine-readable requirements before any dataset goes live.
Scope the dataset inventory against the six categories
The public body lists the datasets it holds and maps each one to the six high-value categories: geospatial, earth observation and environment, meteorological, statistics, companies and company ownership, and mobility. DeployOpen works with the data owners to record the licence status, the update frequency and the format of each dataset, and flags any dataset that cannot be published under CC0 or CC BY 4.0. The step closes with a signed inventory and a written scope. Durations depend on how complete and well documented the inventory is, so the timeline should be agreed in the written scope before work starts.
Decide hosting within the public body's infrastructure
The team confirms where the portal will run: the body's own data center, a government cloud tenancy or an air-gapped network. The decision covers data residency, network access for publishers and the public, backup location and who holds administrator credentials. DeployOpen documents the hosting design and the security controls that apply to it. The step is complete when the hosting decision and its owner are recorded in the scope document.
Install CKAN with PostgreSQL, Solr and Redis
Engineers install CKAN from source on Python 3.10 or newer, with PostgreSQL 12 or newer, Solr 9 (recommended) or Solr 8 for search, and Redis as the cache and job backend. Production deployments run uWSGI and Nginx in front of the application. The team chooses a CKAN minor version and records it, since patch releases are supported only while they are the latest patch. The step closes when a staging instance serves the portal, search returns results and the install is documented in a runbook.
Configure the DataStore and XLoader
Engineers create the separate DataStore database with a write user for loading and a read-only user for the public Data API. XLoader is installed and configured to load uploaded or linked files into the DataStore, since it is the officially supported loader and DataPusher is legacy. Test files covering each high-value category are loaded and queried through filtering, full-text search and pagination. The step is done when the Data API responds for each test resource and the read-only user cannot write.
Map metadata to DCAT-AP with ckanext-dcat
The team maps the inventory fields to the CKAN dataset schema and configures ckanext-dcat for the DCAT-AP version required by the national or European catalog in use. The mapping covers titles, publishers, licences, temporal and spatial coverage, and distribution formats. Engineers check that RDF endpoints produce valid output for sample datasets. The step closes with a mapping table signed by the data owners and a set of validated RDF exports.
Test APIs and bulk downloads
Testers call the CKAN API and the DataStore API for each sample dataset, confirm that responses are machine readable and that licence fields show CC0 or CC BY 4.0. They download the bulk files, compare checksums with the source and check that access is free of registration barriers unless the scope allows otherwise. Load and failure tests confirm that large files and updates do not break the catalog. The step is complete when a written test report lists each result and any defect has been fixed or accepted.
Run a controlled first publication
The public body selects a small set of datasets, one per category where possible, and publishes them in a defined window with named reviewers from the data owner and the communications team. DeployOpen monitors the portal, the search index and the DataStore during the launch and records any incidents. Publishers then confirm that harvesters and external catalogs can read the RDF output. The step closes with a publication log, a go or no-go record and a handover of the operations runbook.
Questions public bodies ask before publishing high-value datasets
What does a CKAN open data portal for high-value datasets cost to run?
The cost splits into one-time work and recurring work. One-time items include scoping the dataset inventory, configuring CKAN, the DataStore and the DCAT-AP mapping, loading and validating the first datasets, and any integration with existing sources. Recurring items include hosting, backups, patching, monitoring and dataset refreshes. The open-source software itself carries no license fee, because CKAN is licensed under the GNU Affero General Public License v3.0. Labor, infrastructure and support are the lines that move. Ask any provider for a quote split into these lines, with the hosting location, support hours and the named source of each recurring item. A written fixed scope lets you compare offers on the same basis.
How long does it take to publish the first high-value datasets?
The timeline depends less on the software than on the readiness of your data. The main drivers are how many of the six categories you hold, how many data owners must be consulted, how clean and documented each source is, and whether files must be converted for the DataStore. Platform setup is usually the shortest part. Ask for a timeline in the written scope with milestones for inventory review, licence decisions, metadata completion, loading, and validation. Use the Commission Implementing Regulation (EU) 2023/138, which applies from 9 June 2024, as the reference for which datasets must be available, and plan backwards from that date.
What must our public body provide before implementation starts?
Most delays come from missing decisions, not missing software. Before work starts, the body should name an owner for each dataset in scope, confirm which of the six high-value categories each dataset falls under, and decide the licence for each dataset. Metadata must also be available: titles, descriptions, update frequency, contact points, and spatial or temporal coverage where relevant. Access to source systems, export formats and a point of contact for data quality questions are also needed. Confirm these inputs in the statement of work so that responsibilities are clear on both sides.
Can our own staff run a CKAN portal after launch?
Yes, if the team has the skills and capacity. Running CKAN requires administration of Python, PostgreSQL, Solr, Redis and a web server such as Nginx, along with regular upgrades. CKAN publishes minor releases about twice a year and supports the latest patch of the current minor version and the previous minor version, so an upgrade calendar is essential. Patch releases do not break backward compatibility, but minor releases may. Decide before launch whether your staff will own patching, backups, dataset publication and incident response, or whether a retainer will cover part of that work. Staff time is the cost most often underestimated.
How do CC0 and CC BY 4.0 licences affect re-use of our data?
Under Commission Implementing Regulation (EU) 2023/138, high-value datasets must be available for re-use under CC0 or CC BY 4.0, or under an equivalent or less restrictive open licence. CC0 places the work in the public domain as far as the law allows, so re-users need no attribution. CC BY 4.0 requires re-users to credit the source. For a public body, CC BY 4.0 keeps attribution visible, which many agencies prefer, while CC0 removes friction for downstream services. Record the licence per dataset in the metadata, because a single portal often holds datasets under different licences. Have legal counsel confirm the choice for each dataset, especially where third-party rights or personal data are involved.
How does DCAT-AP validation fit into the publishing workflow?
Validation should run as part of publication, not as a one-off check at the end. The ckanext-dcat extension supports DCAT-AP v1, v2 and v3 schemas, exposes DCAT RDF endpoints and includes an RDF harvester for importing from other catalogs. Test each dataset's exported RDF against the DCAT-AP version your national or European catalog expects, and fix missing mandatory fields before the dataset is released. Confirm with the receiving catalog which profile and version it accepts. Validation adds effort to the metadata stage, so budget time for data owners to complete fields they may not have filled in before.
Does NIS2 apply to our body, and how would it change the work?
NIS2 under Directive (EU) 2022/2555 covers central government entities regardless of size and lists public administration as a sector of high criticality. Member States may extend it to local public administration, and it does not apply to entities working in national security, public security, defense or law enforcement. Whether your body is in scope depends on your national transposition and your classification, so confirm this with your national authority or legal team first. If in scope, the portal becomes part of the body's risk management and incident reporting obligations, which adds work for logging, access control, vulnerability management and supplier oversight. Scope this with the provider before the fixed price is agreed.
How the scoped work protects the publication timeline
DeployOpen plans, implements, integrates, hardens, documents and supports self-hosted CKAN inside your own infrastructure, whether that is your cloud account, a government data center or an isolated network. The work covers the pieces that determine whether datasets reach the portal on time: the CKAN core, the DataStore with its separate write and read-only database users, the file loader (XLoader rather than the legacy DataPusher), the ckanext-dcat extension for DCAT-AP output, and the harvesters and publishing roles your departments will use. DeployOpen also prepares the evidence your compliance and audit teams will ask for, such as configuration records, access reviews and test results. DeployOpen does not issue audit reports, attestations or certifications. Independent auditors and certification bodies do that work, so the scope should name the evidence package and not promise an outcome.
Engagements are fixed-scope, and scope, deliverables and timelines are agreed in writing before work starts. For a directive-driven deadline, that written scope is the main timeline control. Ask for the list of high-value dataset categories to be published at launch, the named datasets within each category, the API and bulk download targets, the licence each dataset will carry (CC0 or CC BY 4.0 under Commission Implementing Regulation (EU) 2023/138), and the acceptance tests that show each dataset is machine readable and free of charge. Confirm which items are excluded, such as data cleansing at source or migration from an existing portal, because those tend to move dates more than software does. Ask how change requests are priced and approved during the build.
Ongoing work after go-live is a separate retainer, not part of the initial fixed scope. CKAN ships two minor releases a year, and only the latest patch of the current minor version and the previous minor version receive security and bug fixes. Confirm in the retainer which CKAN minor and patch level you will run, who applies patch releases and how quickly, and how a minor upgrade will be tested against your dcat mappings and DataStore loads before it reaches production. Agree in writing who monitors the harvesters and the XLoader queue, and who responds when a scheduled upload fails. Those decisions protect the publication timeline after launch, and they belong in the contract before the first dataset goes live.
How DeployOpen scopes, delivers and supports the portal
DeployOpen plans, implements, customizes, integrates, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure. For an EU Open Data Directive portal, that means agreeing in writing which high-value dataset categories, licences, APIs and metadata outputs are in scope before any build work starts.
How we scope
- Inventory which datasets fall within the six high-value categories and which are already published elsewhere.
- Confirm the licence per dataset, targeting CC0 or CC BY 4.0 under the implementing regulation.
- Agree the hosting location, identity model and DCAT-AP output in a written, fixed-scope statement of work.
How we build
- Install CKAN from source with PostgreSQL, Solr, Redis and a DataStore read-only user for public API access.
- Configure XLoader and ckanext-dcat so tabular resources load into the DataStore and metadata exports as DCAT-AP.
- Test bulk download, API queries and RDF endpoints against the agreed scope before any dataset goes live.
How we hand over
- Document the deployed architecture, configuration, dataset publication workflow and upgrade procedure for your team.
- Deliver evidence of controls and test results so your assessors can trace each requirement to a configured setting.
- Walk your administrators through backup, restore and CKAN patch-release handling before final acceptance.
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer after go-live, scoped separately in writing.
- Track CKAN minor and patch releases so your portal stays on a supported version within the project's policy.
- Confirm with us in the quote which support hours, response processes and upgrade responsibilities apply to your environment.

