Open data portal requirements checklist

Detection is a workflow step you design, not a feature you switch on, so write it into requirements as a gate that runs before any dataset becomes public. Require a documented scan of titles, descriptions, resource files and DataStore columns for identifiers such as names, email addresses, national ID numbers and postal addresses, using rules your privacy lead has approved. Make the gate block publication by default and record who cleared each exception. To test it, load a sample dataset with planted dummy identifiers, including values in a CSV column, a free-text description and a resource file name, then confirm the publish step stops and logs the finding. Ask any implementation partner to show the test result, not a description of the scanner.

Write retention periods for drafts, deleted datasets and their resource files as separate rules, because a portal often keeps them in different places. Your records or legal team sets the periods; the requirement is that each period is written down, enforced by the platform or a scheduled job, and visible in logs. Confirm how the CKAN version you choose handles deletion, purging and removal of files and DataStore tables, since a soft delete may leave data in the database and search index. To test it, create a sample draft and a sample dataset with an uploaded file, delete both, then check the database, file storage, search index and backups against the written schedule. Record what remains and for how long.

Separate them with access rules set per dataset and per organization, not by trusting staff to choose the right folder. Require that restricted datasets are never exposed through the public catalog, search, API, DataStore endpoints or DCAT output, and that the classification is a required field that the publishing workflow checks. Also check derived copies, such as previews, exports and harvested metadata, since a restricted resource can leak through them. To test it, create one open and one restricted sample dataset with identical contents, then query each through the web interface, the API without a token, the DataStore read endpoint and any RDF output as an anonymous user and as a user outside the owning organization. Any leak fails the requirement.

Write requirements for what is logged, where it is stored, how long it is kept and who can read it. At minimum, logs should cover logins, failed logins, dataset creation, edits, deletions, publication changes, permission changes and API token use, with timestamps and the acting account. Store logs outside the application server so an administrator cannot silently edit them, restrict read access to a small named group, and set a retention period your records team approves. Confirm whether the CKAN configuration and any reverse proxy or identity provider write these events by default, because gaps are common. To test it, perform one of each action with a sample account, then check that each event appears in the log store with the correct user and time.

Define the approval chain in writing before launch: who may create and edit a dataset, who may request publication, and who has the authority to approve it. Most organizations need at least a separation between the publisher and the approver, so one person cannot release a dataset alone. Record the approval in the portal, with the approver's identity, date and the version of the metadata and resources approved. Changes after approval should return the dataset to review unless a documented minor-change rule applies. To test it, run a sample dataset through the full path with two sample accounts, attempt to self-approve, and confirm the platform or workflow blocks it and keeps an audit trail.

Legal interpretation belongs to your counsel and data protection officer, and no portal platform or implementation partner can settle it for you. Ask them to confirm the legal basis for publishing each dataset class, the retention periods that apply, whether any dataset requires anonymization or aggregation before release, and which access rules apply to staff and to the public. Translate their answers into numbered requirements with an owner, a test and an acceptance date. Confirm also whether a data protection impact assessment is required for the portal itself. Keep their written decisions with the requirements document so the build can be audited against them later.

Your team can write the rules, and most of the work is defining decisions only your organization can make, such as classifications, retention periods and approvers. A partner adds value in turning those decisions into configured controls, scanning rules, logging setup and repeatable tests, and in documenting the evidence. Cost depends on how many dataset classes, organizations and approval paths you have, how much custom scanning you need, and how many test cycles are required before sign-off. Ask any vendor to quote against a written scope listing these controls and tests, and to state which items they will deliver and which your staff must complete.

The total depends on four cost lines: hosting, implementation, support and internal staff time. CKAN itself is licensed under the GNU Affero General Public License v3.0, so there is no license fee for the core platform, but the other lines are real. Hosting is paid to your cloud provider or data center and recurs every month. Implementation is a one-time cost paid to whoever installs, configures and integrates the portal. Support and upgrades recur, and they rise with the number of extensions, harvesters and integrations you run. Internal staff time is often the largest hidden line, because data owners must publish and maintain datasets. Ask any partner for a quote that splits these lines separately, states what is one-time and what recurs, and names the assumptions behind each.

Duration depends mainly on scope, not on the software. The main drivers are the number of metadata profiles you must support, how many existing data sources need migration or harvesting, whether you need SSO for staff, how many publishing organizations must be onboarded, and how much accessibility testing is required. Installation from source is a defined technical task, and CKAN's install guide lists PostgreSQL, Solr, Redis, uWSGI and Nginx as components to provision. Your hosting approvals and security reviews often set the pace more than the build. Ask for a timeline tied to named milestones and to the deliverables in the written scope, not a single end date.

Before work starts, your team should prepare four inputs. First, named data owners for each publishing organization who can approve datasets and licenses. Second, a set of sample datasets in the formats you actually publish, including large files that will be loaded into the DataStore. Third, SSO details for staff accounts, such as the identity provider, the protocol in use and the attributes you need to map. Fourth, hosting access: cloud accounts or data center credentials, network paths, DNS, certificates and backup targets. Missing any of these usually moves dates, so list them as prerequisites in the statement of work.

Yes, a capable platform team can install CKAN from source, and the official documentation covers Ubuntu 20.04 and 22.04 64-bit, with source installs also working on Red Hat, Fedora and CentOS. The harder work is ongoing: tracking CKAN's release policy, applying patches, handling minor releases that may include backwards-incompatible changes, and keeping extensions compatible. CKAN supports the latest patch of the current minor version and the previous minor version, so you must plan upgrades on a schedule. Assess whether your team can keep that cadence, handle security hardening and run tested backups before deciding to build alone.

Define scope by listing deliverables, not features. A workable scope names the CKAN version and extensions, the metadata profile and whether it exports DCAT-AP or another schema, the number of organizations and user roles, the integrations (SSO, harvesters, analytics), the DataStore loading method, the hosting environment, the accessibility target, the backup and restore tests, the training sessions and the handover documents. Name what is excluded, such as data cleaning or content migration, and state the acceptance criteria for each item. DeployOpen agrees scope, deliverables and timelines in writing before work starts, so ask any partner to show the same level of detail.

Start with the legal requirement, then test the architecture against it. Confirm where the portal, its PostgreSQL database, Solr index, Redis cache, backups and logs will run, and where any third-party services such as email, CDN or monitoring send data. Because CKAN is self-hosted, you control these locations, but extensions and external services may not be. Check the DataStore, which stores structured resource data in a separate PostgreSQL database, and the harvesters, which pull data from other catalogs. Ask your legal or data protection team to confirm the rules, then request a written data flow diagram from any partner and verify it during acceptance testing.

Extensions change both the data flow and the compliance evidence. The ckanext-dcat extension provides DCAT-AP v1, v2 and v3 schemas, RDF endpoints and an RDF harvester, and it is supported on CKAN 2.10 and 2.11, so confirm compatibility with the CKAN version you choose. The ckanext-saml2auth extension is one option for SAML-based SSO, but it is a third-party project, so review its code, maintenance status and fit with your identity provider before relying on it. Each extension adds upgrade work, so list them explicitly in the scope.