What a CKAN implementation involves
Implementing CKAN means standing up a self-hosted data management system in infrastructure you control, then configuring it for your metadata, your publishers and your data flows. The core stack is a CKAN web tier (Python, with uWSGI and Nginx recommended for production), a PostgreSQL database for the catalog, Solr for search and Redis for caching and background work. A separate PostgreSQL database backs the DataStore, which holds structured resource data with a write user and a read-only user. Each component has its own sizing, backup and patching responsibilities, so the work is more than installing one application.
Most projects start from one of three triggers: a new open data mandate that requires a public catalog, the need to replace a spreadsheet or static catalog that no longer scales, or the consolidation of several departmental portals into one. The trigger shapes the plan. A mandate usually fixes a deadline and a metadata standard, while consolidation adds the harder work of mapping existing catalogs, owners and naming conventions. Write the trigger, the required metadata profile and the list of publishers down before choosing a design, because they decide how much schema and onboarding work follows.
Teams can build a CKAN portal in-house or with a partner. Teams with strong Linux, PostgreSQL and Python operations and a clear metadata owner often build the platform themselves and bring in help for specific gaps. A partner is most useful in three places: the architecture decisions (hosting topology, Solr and DataStore sizing, which extensions to depend on), the selection and review of extensions such as ckanext-dcat for DCAT-AP profiles, and the first publisher cohort, where workflow and data quality problems surface. Ask any partner to show how they will hand over runbooks and who will patch the portal after go-live.
Confirm early which CKAN release you will target. Versions and support windows change, so check the current release notes and the release policy before you commit to a version. Also confirm with your platform team the Python, PostgreSQL and Solr versions that the chosen release requires, and whether your operating system is covered by the install documentation or will need a source install. These decisions feed directly into the scope and quote, so settle them before the statement of work is written.
What the work includes
A CKAN implementation scope usually breaks into the work streams below. Each one ends in a named deliverable, so the quote can be checked line by line against what you will receive.
Environment and hosting design
Decide the topology for web, worker, database, search and cache nodes inside your cloud accounts, data centers or air-gapped network, along with network zones, backups and environments for staging. The deliverable is a written architecture and environment specification you can approve before build. Check who owns cloud accounts, DNS and certificates.
CKAN installation and service configuration
Install CKAN from source on a supported operating system, with uWSGI and Nginx for production, and set the site configuration, plugins and service units. The CKAN install guide covers Ubuntu 20.04 and 22.04, and the source install requires Python 3.10 or newer. The deliverable is a configured instance that passes smoke tests in staging and production.
PostgreSQL, Solr and Redis setup
Provision the main PostgreSQL database, the Solr search index (Solr 9 is recommended) and Redis, and size each for your dataset volume and query load. Decide whether these run on virtual machines, containers or managed services in your environment. The deliverable is a tested connection and index configuration with documented ports and access rules.
DataStore with separate write and read-only users
Create the DataStore database as its own PostgreSQL database with a write user for CKAN and a read-only user for the Data API and SQL queries. Set permissions so public queries cannot change data. The deliverable is a verified role and permission configuration, with tests that confirm the read-only user is refused on write statements.
XLoader configuration for resource loading
Configure XLoader, the officially supported loader, to push uploaded and linked files into the DataStore, and set file size limits, timeouts and job monitoring. DataPusher is legacy, so do not plan new work around it. The deliverable is a tested loading pipeline with a sample of CSV and spreadsheet resources loaded end to end.
Metadata schema and DCAT profile decisions
Decide which metadata fields, required vocabularies and DCAT-AP profile version your portal will publish, using ckanext-dcat for DCAT-AP v1, v2 or v3 schemas and RDF endpoints. Confirm which profile your regulator or aggregator expects. The deliverable is a metadata schema specification with field definitions, validation rules and an example record.
Harvester setup for external catalogs
Configure harvesters to import datasets from other catalogs, including the RDF harvester from ckanext-dcat, and set schedules, mapping rules and duplicate handling. Agree with source owners how deletions and updates should flow. The deliverable is a set of tested harvest sources with a run log and a documented mapping for each source.
Theme work and portal customization
Scope the branding, templates, layouts and any custom extension work needed beyond the default look, and define accessibility checks for the templates. Keep custom code in a versioned extension so upgrades stay manageable. The deliverable is a theme extension in your repository with a design sign-off and a rendered page checklist.
Publisher onboarding and training
Define how publishing organizations are created, which roles they receive, and how their datasets are reviewed before release. Train a named group of publishers and administrators on the workflow in your environment. The deliverable is a trained publisher group, role matrix and onboarding checklist that your team can repeat for new organizations.
Handover documentation and runbook
Produce the operating documentation your team needs after go-live: install and configuration records, backup and restore steps, Solr and DataStore operations, patch procedure and escalation contacts. The deliverable is a runbook and architecture record stored in your own documentation system, with a handover session that tests the procedures.
What drives cost and duration
Each row is a driver that belongs in a CKAN quote, why it moves effort, and how to keep it down before work starts.
| Driver | Why it moves effort | How to keep it down |
|---|---|---|
| Publishers and datasets | Each publishing organization needs accounts, roles, onboarding, and checks on how its records are entered. Effort rises with the number of organizations and with how many datasets need manual review before publication. | Start with the publishers ready now, onboard the rest in waves, and give each publisher a single named contact who submits records in agreed templates. |
| Tabular resources for the DataStore | Files loaded into the DataStore must parse cleanly. Large files, many sheets, merged cells, mixed types, and inconsistent headers each add mapping and cleanup work and slow loads. | Limit the first release to a defined set of files, standardize on CSV with one header row, and agree on column types before upload. |
| Harvest sources and formats | Each harvest source needs its own configuration, field mapping, and error handling. Sources that publish in different formats, such as RDF, JSON, or custom APIs, need more testing than sources that share one format. | Inventory sources early, rank them by value, and launch with the fewest sources that meet the first release goal. Ask for a mapping document per source as part of the quote. |
| Metadata profile complexity | Choosing a DCAT-AP version, adding custom fields, and setting validation rules affects the schema, the templates, the RDF export, and the publisher forms. The ckanext-dcat extension supports DCAT-AP v1, v2 and v3, so the chosen version drives the scope of mapping work. | Limit the first release to a core schema, confirm the required DCAT-AP version with your reviewers, and defer optional custom fields to a later phase. |
| Theming depth | A standard look needs less work than a branded design with custom templates, page layouts, and accessibility checks. Each custom template adds build, test, and upgrade effort for later CKAN releases. | Reuse the base theme with brand colors, logos, and text changes in the first release, and log any template overrides so they can be reviewed at each upgrade. |
| Number of environments | Development, test, staging, and production each need their own install, data, Solr index, and configuration. Keeping them aligned takes ongoing work, and extra environments add setup and test cycles. | Agree on the minimum set of environments the project needs, use the same deployment method for each, and seed test environments from anonymized or sample data. |
| Hosting model | Cloud accounts, a data center, or an air-gapped network each bring different provisioning, image handling, update delivery, and access controls. Air-gapped installs need offline package and container sources and a manual update path. | Confirm the hosting model before scoping, and ask the network and security teams to provide access paths, approved images, and change windows in writing. |
| Identity integration | Single sign-on with ckanext-saml2auth requires identity provider metadata, attribute mapping to organizations and roles, and testing with each identity provider. Local registration and sysadmin management add further rules. | Confirm which identity provider and attributes will be used, decide early whether local accounts stay enabled, and provide a test identity provider or test accounts. |
| CKAN minor version | Each minor release can include backwards-incompatible changes, and extensions must support the chosen version. Starting on a version that is close to end of support adds upgrade work soon after launch. | Confirm the current supported minor versions against the CKAN release policy before scoping, and choose the newest supported minor version that all required extensions support. |
Project phases and outputs
The sequence moves from agreed metadata and a hosting decision to a tested staging portal, onboarded publishers and a handed-over production system.
Run discovery and metadata design
DeployOpen works with your data governance lead, publishers and existing catalogs to inventory dataset types, required metadata fields, licensing, and access rules. The step closes with a signed-off metadata schema and a list of open questions, so the build scope is fixed before any server is provisioned.
Record the architecture and hosting decision
The team compares hosting options in your own cloud accounts, data centers or air-gapped network against your security and operating constraints, and sets the topology for PostgreSQL, Solr, Redis, uWSGI and Nginx. The output is an architecture decision record that names the chosen CKAN minor version, supported Python and Solr versions, and the team that will own each component.
Build development and staging environments
DeployOpen provisions the development and staging environments in your infrastructure, installs CKAN with its database, search and cache services, and sets up configuration as code and backups. The step is done when a staging instance runs the agreed version, passes a smoke test of login, dataset creation and search, and can be rebuilt from the repository.
Select and configure extensions
Based on the metadata schema, the team chooses extensions such as ckanext-dcat for DCAT profiles and harvesting, DataStore with XLoader for tabular resources, and ckanext-saml2auth if single sign-on is required. Each extension is checked for compatibility with the chosen CKAN version and reviewed for security. The output is a configured staging instance and a written list of approved extensions with their versions.
Apply theming and access configuration
DeployOpen applies your branding and templates, sets organization and role permissions, and configures the DataStore read-only database user and API token policy. The step closes when a reviewer has checked the portal against the approved design and the permission model, and the findings are logged and resolved.
Load data and test harvesting
Sample and then production-representative datasets are loaded, DataStore uploads are processed through XLoader, and harvest jobs are run against each source catalog. The step is complete when harvest runs produce the expected records, failures are logged with clear reasons, and the test results are recorded against the metadata schema.
Onboard publishers and run a pilot
A small group of publishers is trained, given accounts and organization roles, and asked to publish real datasets in staging. Their feedback is turned into configuration or documentation changes. The pilot closes with a publisher guide and a list of accepted changes that your team has approved.
Cut over to production and hand over
DeployOpen runs the production cutover in a planned window, verifies backups and monitoring, and hands over the runbooks, configuration repository, upgrade procedure and support contacts to your operations team. The step is done when the production checks pass and your team has accepted the handover in writing. Ask for a dated plan with each phase in the written scope.
What to keep in-house and what to outsource
The table separates governance work that a public body should keep, from platform engineering often outsourced, and lists the skills each part needs and which are scarce in a typical public body.
| Part of the work | Who usually owns it | Skills the part needs | Scarce in a typical public body? |
|---|---|---|---|
| Data governance and publisher ownership | In-house | Open data policy, records and licensing knowledge, stewardship across departments | No, usually present, but spread across teams with no single owner |
| Metadata policy and schema decisions | In-house, with partner input | Metadata standards knowledge (for example DCAT-AP profiles), vocabulary design, controlled lists | Yes, few staff know both the standard and the local publishing practice |
| Day-to-day content approval and publisher support | In-house | Editorial judgment, data quality review, helpdesk habits for publishers | No, but needs time and a named owner |
| Platform architecture (hosting, PostgreSQL, Solr, Redis, Python web stack) | Often outsourced or shared | Linux operations, Python web operations, PostgreSQL administration, container or VM design | Yes, these skills are often absent or limited to one administrator |
| Solr and PostgreSQL tuning | Often outsourced | Search tuning, index sizing, query analysis, PostgreSQL administration and backup testing | Yes, rarely held in-house outside large teams |
| CKAN extension configuration (for example ckanext-dcat, ckanext-saml2auth) | Often outsourced, then handed over | Familiarity with CKAN plugin configuration, extension compatibility checks, Python debugging | Yes, few staff have worked on CKAN internals |
| Theming and front-end work | Often outsourced | Front-end development, CKAN template customization, accessibility review | Yes, most public bodies lack in-house front-end capacity for CKAN templates |
| Harvester engineering | Often outsourced | Metadata mapping, RDF and DCAT parsing, scheduled job operations | Yes, mapping source catalogs needs both standards and CKAN harvest knowledge |
| Cutover and go-live | Shared, planned with partner | Migration scripting, DNS and reverse proxy changes, rollback planning, test sign-off | Partly, the in-house team needs to own sign-off and rollback decisions |
What usually goes wrong
Each card names a failure mode in a CKAN implementation and the step that prevents it. Most of these are cheap to avoid at planning time and expensive to fix after publishers and users depend on the portal.
Version, runtime and extension drift
Choosing a CKAN minor version outside the supported window, or a Python runtime older than 3.10, forces a rebuild later. Extensions also drift when development and production pin different versions. Prevention: check the release policy before choosing the target, pin CKAN, Python, PostgreSQL and every extension in version control, and deploy the same locked set to every environment.
Underestimated Solr and Redis operations
Solr 9 is recommended, Solr 8 is also supported, and Redis is required. Teams often treat both as set-and-forget services, then discover index rebuild time, memory pressure and version coupling only during an upgrade. Prevention: name an owner for each service, run a documented index rebuild in staging, and put search index backup and monitoring into the plan.
Wrong loader for large files
XLoader is the officially supported tool for loading files into the DataStore, and DataPusher is legacy. Sending large or badly formatted CSV and spreadsheet files through the wrong path produces failed or partial loads that look like portal bugs. Prevention: confirm XLoader in the target design, test the largest and most awkwardly formatted sample files in staging, and define how publishers learn that a load failed.
Metadata schema publishers cannot fill
A schema copied from a full standard vocabulary, with many mandatory fields, produces empty records and stalled publishing. Map each field against what publishers can supply from their own systems, and mark as required only the fields the portal needs for its purpose. If you adopt DCAT profiles through ckanext-dcat, test the field list with a pilot group of publishers before locking it.
Duplicate or inconsistent harvested records
Harvesting from several source catalogs can create the same dataset under different identifiers, or copies that drift from their source. Before go-live, decide the identifier and update rule for each source, decide which fields the portal owns, and review a deduplication report after the first full harvest. Confirm harvester behavior on your CKAN version, since ckanext-dcat is supported on CKAN 2.10 and 2.11.
Onboarding before the workflow is settled
Inviting publishers before approval, review and correction steps are fixed means users receive changes under rules that later shift. Document the publish, review and correction workflow, the organization and role setup, and the support contacts first. Pilot with a small group, then open onboarding in waves once the workflow has run through a full publish and correction cycle.
How the work changes by starting point
A new portal with no existing catalog has the simplest migration scope, but the metadata schema, publisher roles and DataStore design still need decisions before the build starts. Most of the effort moves into planning and onboarding.
- Scope: schema decisions, organization and publisher structure, theming and the first set of publisher workflows.
- Effort: driven by how many publishing organizations exist and how much custom metadata they need, not by data volume.
- Dependencies: a named owner for the metadata standard and a decision on whether DCAT output is required.
- Confirm before quoting: the number of publishers, whether any resources will be loaded into the DataStore, and the hosting target.
Moving datasets from spreadsheets or another catalog adds a data mapping stage. The work is in cleaning, mapping fields to the CKAN schema and checking results, so the source quality controls the timeline more than the platform does.
- Scope: field mapping, import scripts, validation rules and a reconciliation report against the source.
- Effort: rises with inconsistent source fields, duplicate records and any harvesting from remote catalogs.
- Dependencies: access to source exports, a data owner who can resolve conflicts and agreement on which fields are mandatory.
- Confirm before quoting: the number of source files or catalogs, whether the source uses a standard such as DCAT, and who cleans the data.
Replacing a legacy CKAN 2.x portal moves the work from a fresh build into a migration of data, custom extensions and infrastructure. The migration page covers the detailed steps, so this tab describes only how the scope shifts.
- Scope: upgrade path to a supported release, review of each custom extension and cutover planning.
- Effort: driven by the number of custom extensions and how far the portal sits behind the current release.
- Dependencies: access to the existing database, Solr index and extension source code.
- Confirm before quoting: the exact CKAN version in production, the list of installed extensions and whether a parallel environment is allowed. See the CKAN migration and portal upgrades page for detail.
An Ubuntu 22.04 installation with Docker-hosted Solr and Redis follows the supported package install path. The Docker services remove part of the host setup, but the team still owns the container hosts, volumes and upgrades.
- Scope: host preparation, PostgreSQL, Python 3.10 or newer, Solr 9 or Solr 8 in containers, Redis and uWSGI with Nginx for production.
- Effort: lower than a source build on an unsupported distribution, with most time spent on configuration and testing.
- Dependencies: a Docker host policy, persistent volumes for Solr and a decision on who patches the container images.
- Confirm before quoting: whether the Docker images are permitted in your environment, the Solr version you intend to run and how the container images are sourced and scanned.
A source install on Red Hat, Fedora or CentOS is supported by the project's source guide, but the packages, service management and Python setup differ from the Ubuntu path. Expect more time on dependency resolution and system hardening.
- Scope: source install of CKAN, PostgreSQL, Solr, Redis and the web stack, with service units written for the target OS.
- Effort: rises with package version gaps between the distribution and the required Python, PostgreSQL and Solr versions.
- Dependencies: the OS release your organization supports, approved repositories and SELinux or firewall policy.
- Confirm before quoting: the exact OS release, whether packages come from vendor repositories or from source, and who owns OS patching after go-live.
An air-gapped or restricted network changes delivery more than design. Every package, container image and extension must be moved in by approved means, and updates follow the same path, so the plan has to include an offline supply chain.
- Scope: an artifact inventory, offline mirrors for OS packages, Python dependencies and container images, and an update procedure.
- Effort: driven by the number of dependencies to mirror and the approval steps for each transfer.
- Dependencies: a transfer process, an internal package repository and no runtime calls to external services for harvesting or themes.
- Confirm before quoting: which external services are blocked, how new releases reach the network and who approves imports.
What the client needs to provide
The decisions that must be settled before the build starts are the ones that change the architecture: where CKAN runs, who owns it, and which metadata profile the portal must follow. Hosting accounts or servers, network paths to the systems the portal will talk to, and outbound access for harvesting need to be in place before the first environment is built. Confirm with your cloud or data center team the hosts for the web application, PostgreSQL, Solr and Redis, and whether each will run in a separate tier. Name a technical owner who can approve configuration and a governance owner who can sign off on metadata, publishing rules and who may become a sysadmin.
Metadata requirements should be fixed early. State whether you must follow DCAT-AP or a national profile, which mandatory fields apply, and which controlled vocabularies you will use. Also agree on the sample datasets and file formats you expect to load into the DataStore, such as CSV or XLSX, because these determine the XLoader testing scope and expected file sizes. Provide a list of publishers with a named contact each, and the list of harvest sources with URLs, formats and any credentials, so the team can plan which harvesters and RDF endpoints are needed.
Some decisions can wait until after the pilot: final theme and layout beyond the first release, the full publisher rollout order, and advanced harvest schedules. Branding assets such as logos, color values and footer text should be ready before theming starts, but the broader design can change once users see the pilot. Identity and registration choices should be made by the start: decide whether the portal uses local accounts, an identity provider through SAML2, or both, and whether public self-registration is allowed. Ask your identity team to confirm which attributes will map to organizations and roles, since this affects configuration and testing.
Frequently asked questions
What drives the cost of a CKAN implementation, and is hosting included?
Scope drives most of the cost. CKAN has no license fee under the GNU Affero General Public License v3.0, so a quote covers the work and the infrastructure you run it on. The main drivers are the number of publishers to onboard, the metadata schema (for example, a DCAT-AP profile through ckanext-dcat), custom extensions and theming, the volume of files loaded into the DataStore, and SSO and harvesting integrations. Hosting is not included in the sense of DeployOpen running servers for you. DeployOpen builds the portal inside your cloud accounts or data centers, so the compute, storage and network you pay for remain yours. Confirm whether provisioning appears as a line item in the quote. The effort drivers section lists the quote items.
How long does a CKAN implementation take, and what sets the timeline?
Timing depends on scope, and no single duration fits every portal. A fixed-scope agreement sets the schedule in writing before work starts, so ask for milestones tied to named deliverables. The biggest timeline drivers are the number of custom extensions and theme changes, the volume and condition of the data to load, how many publisher groups need onboarding and training, the time your identity team needs to configure SAML2 single sign-on, and how quickly your infrastructure team provisions servers, databases and networking. Approval cycles for metadata policy add time that no build plan removes. Ask the partner which milestones depend on inputs from your team.
Can our own team run a CKAN implementation without an outside partner?
Yes, if your team has the skills for the stack. A source install needs Python 3.10 or newer, PostgreSQL 12 or newer, Solr 9 (recommended) or Solr 8, and Redis, with uWSGI and Nginx recommended for production. Your staff also need to cover publisher and metadata decisions, infrastructure provisioning, identity administration for SSO, and ownership of backups and upgrades. Expect an ongoing commitment after launch, because CKAN targets about two minor releases a year and your extensions must keep pace. Before deciding, ask a partner or your own architect to review these skills against your environment.
Which CKAN version should a new portal start on today?
Start on the latest patch of the current minor release, which is CKAN 2.12.0 as released on August 26, 2026, and confirm the newest patch when the build starts. CKAN supports the latest patch of the current minor version and the previous minor version, so 2.11 also receives security and bug fixes, while older patches lose support once a newer patch is out. Patch releases do not break backwards compatibility, but minor releases may. Before you commit, confirm that every extension you need supports the target version. ckanext-dcat is listed as supported on CKAN 2.10 and 2.11, so check its status on 2.12 with the maintainers.
How do XLoader and the DataStore change the build and its cost?
XLoader is the officially supported tool for loading uploaded or linked files into the DataStore, and DataPusher is now legacy, so plan the build around XLoader. The DataStore keeps structured resource data in a separate PostgreSQL database with a write user and a read-only user, and it exposes a Data API for filtering, full-text search, pagination and SQL queries. That adds a database to size, back up and monitor, a loading job to run, and Solr and Redis to keep healthy. Load time and storage grow with file size and count, so share a sample of your largest files and their formats before effort is estimated. Confirm the read-only user setup with your security reviewer.
What does a fixed-scope CKAN implementation quote include?
A fixed-scope quote lists the scope, deliverables, acceptance criteria, assumptions, your responsibilities and the timeline in writing before work starts. The fee follows from that written scope. Expect line items grouped by phase: environment build, PostgreSQL, Solr and Redis setup, metadata schema, DataStore and XLoader configuration, harvesting, theming, extensions, SSO, publisher onboarding and handover documentation. Ask which costs are one-time (the build) and which recur (infrastructure you pay for, patching, upgrades and any support retainer, which is separate). Ask which assumptions would change the price if your data volume or extension list changes. The effort drivers section explains each line item.
How DeployOpen scopes and delivers CKAN
DeployOpen plans, builds and hands over CKAN portals inside infrastructure you control, with the hosting, search, DataStore, metadata and publisher decisions written down before work starts. Engagements are fixed-scope, and ongoing support is available as a retainer.
How we scope
- The written scope lists hosting targets, CKAN version, Solr, Redis and PostgreSQL layout, and extensions to be used.
- Metadata schema, DataStore and XLoader use, harvesting sources and theming are each defined as scoped deliverables.
- Publisher onboarding roles, expected volumes and sign-in requirements are recorded so the quote reflects the actual work.
How we build
- Installation follows the supported CKAN source install path, with Python, PostgreSQL, Solr and Redis versions confirmed against the chosen release.
- The DataStore is configured with separate write and read-only database users, and XLoader is set up for file loading.
- Metadata profiles, harvesters and theming are built and tested against sample datasets in a staging environment before go-live.
How we hand over
- Handover includes architecture documentation, runbooks for upgrades, backups and search index rebuilds, and configuration records.
- Publishers receive onboarding material and the administrator workflow for adding organizations, datasets and harvest jobs.
- A clear list of open items, extension version pins and the patch cadence your team must follow is delivered at the end.
How we support
- Ongoing support is available as a retainer covering patch releases, extension compatibility checks and Solr or Python changes.
- Support scope, response expectations and upgrade work are written into the retainer before it starts.
- Your team keeps a clear view of which CKAN minor version is running and when the next supported upgrade should be planned.
