Research data portal for universities

Universities and research institutes that share datasets need a catalog that separates open from restricted data, records who accessed what, and fits campus identity. This page covers how a self-hosted CKAN portal meets those needs and what to decide before building.

On this page

Why a research data portal for universities is not a generic open data build

A research office or library lead usually arrives with three problems at once: funders now expect a data management plan and sharing statement, the most useful datasets sit on departmental drives nobody else can find, and some of those files also hold student or participant records. CKAN can serve as the catalog, but the build is driven by access tiers, persistent identifiers and student-record law, not by publishing alone. A portal that only lists datasets with download links will satisfy neither a funder reviewer nor a registrar. The short answer on platforms is that CKAN handles the catalog, metadata and access rules for datasets, while learning platforms stay where they are. Moodle and Open edX sit beside the portal, not inside it. Course materials, learner progress and enrolments belong to the LMS and its own privacy controls. The portal should link to course-derived research datasets only after they are de-identified or released under an agreed tier. Confirm with the registrar and the research office which records reach the portal, because FERPA obligations for education records follow those records wherever they are copied. The decisions to make early are concrete: which datasets are open, which are restricted to named researchers, and which can be released only under a data use agreement. Each tier needs its own access rule, and the DataStore read-only user must not expose restricted tables. Decide how DOIs or other persistent identifiers will be minted and recorded in dataset metadata so citations resolve. Finally, check whether campus SSO through SAML, such as the ckanext-saml2auth extension, can map institutional roles to publishing and access rights before any dataset goes live.

Which university requirements shape the portal build

This table maps the US federal rules most likely to touch a university research data portal to specific build decisions in CKAN. Read each row as a question to put to counsel and the security team, since applicability depends on the institution's funding, data holdings and the people whose records appear in the datasets. The main takeaway is that access tiers, audit logging and MFA decisions made early are far cheaper than retrofitting them after datasets are published.

RequirementWhat it asks forBuild decision in CKANDepends on the institution
FERPA (34 CFR Part 99)Education records need access limited to school officials with legitimate educational interests, and each disclosure must be logged with the parties and their interests.Separate restricted and open datasets with organization-level roles, keep student-level records out of public DataStore tables, and enable audit logging of access and downloads.Whether any dataset contains student education records, and whether the institution receives covered federal funds.
Outside contractors as school officials (34 CFR 99.31)A contractor such as a portal implementation partner may see records only under the institution's direct control and under redisclosure limits.Contract terms with the implementer, named admin accounts for partner staff, and removal of their access at project end.Whether the institution uses contractors for this work and how its agreements are written.
Records access within 45 days (34 CFR 99.10)Requests to inspect education records must be met within a reasonable period and no more than 45 days after receipt.Maintain a searchable inventory of which datasets hold personally identifiable information so staff can locate records quickly.Internal request handling process and who owns it.
GLBA Safeguards Rule (16 CFR Part 314)Title IV institutions need a written information security program and multi-factor authentication for anyone accessing an information system.Enforce MFA for all staff and admin logins, integrate campus SSO, and document the portal in the written security program.Whether the institution participates in Title IV programs and how its program is scoped.
FTC notification within 30 daysA notification event affecting at least 500 consumers must be reported to the FTC within 30 days of discovery.Retain portal and DataStore logs, define an incident runbook for data exposure, and test the contact path with the security office.Institution's incident response plan and the number of affected individuals in any event.
COPPA (16 CFR Part 312)Operators collecting personal information from children under 13 need a written security program, annual risk assessments and a retention policy with a deletion timeframe.Avoid collecting child-level data in research datasets, or isolate it with a documented retention schedule and scheduled deletion jobs.Whether any research involves children under 13 and whether data is collected online from them.
ADA Title II web rule (WCAG 2.1 AA)State and local government web content must meet WCAG 2.1 Level AA, with compliance dates of April 26, 2027 or April 26, 2028 depending on entity size.Apply WCAG 2.1 AA page templates to the catalog, dataset pages and forms, and test them before launch.Whether the institution is a public entity covered by Title II, such as a state university, and its population category.
Persistent identifiers and citationFunders and journals expect datasets to be citable, which depends on stable identifiers and metadata that the institution controls.Plan DOI or other identifier integration and metadata templates for citation, confirmed with the chosen registration agency.Which identifier provider the institution already uses and funder policy requirements.

How open and access-controlled datasets sit in one catalog

Fully open research outputs, such as published tables, anonymized survey results and code-linked datasets cleared for release, are the simplest class. In CKAN they are typically public datasets owned by a research group or department organization, and their resources can be read through the web interface, the Data API and downloads without a login. The DataStore is useful here because it lets users filter and query tabular resources instead of downloading whole files.

  • Dataset visibility: Set the dataset to public so it appears in search and in the Data API for anonymous users.
  • Organization ownership: Assign each dataset to the publishing department or lab organization so editors can manage it without access to other groups.
  • DataStore reads: Load tabular resources with XLoader and let anonymous users query them through the read-only DataStore user.
  • Release review: Confirm the release decision, licence and disclosure check are recorded before the dataset is made public, because CKAN does not make that decision for you.
  • Retraction path: Decide who can switch a dataset back to private or delete it, and how the change is communicated.

Check that anonymous API calls return only the resources you intend to publish, and that a dataset flipped to private disappears from both search and the Data API.

Which platforms fit a university research data catalog

A research data catalog, a teaching platform and a project website solve different problems, and each needs a different open-source tool. The cards below separate the dataset catalog from the systems around it, so teams can choose the right platform for each job and confirm the extensions they depend on.

CKAN as the primary catalog

CKAN is the open-source data management system built for data hubs and portals, licensed under AGPL-3.0. For a university, it holds dataset records, resource files and access settings for each research group or institute. Before you commit, check that your AGPL-3.0 obligations are acceptable to your legal team, and confirm which CKAN minor version your chosen extensions support, since CKAN ships about two minor releases a year.

DataStore for tabular research data

The CKAN DataStore keeps structured data from resources in a separate PostgreSQL database, with a write user and a read-only user, and exposes a Data API for filtering, search and SQL queries. This matters for survey tables, instrument readings and longitudinal datasets that researchers want to query without downloading full files. Plan which datasets are loaded, and confirm that XLoader, not the legacy DataPusher, is used for uploads.

DCAT metadata for funders and national catalogs

The ckanext-dcat extension provides DCAT-AP v1, v2 and v3 schemas, RDF endpoints and an RDF harvester. It lets the catalog export metadata that funder repositories and national catalogs can read, and import records from other catalogs. Map your funder metadata fields to DCAT terms early, and confirm which DCAT-AP profile your funder or national catalog expects, because the extension is supported on CKAN 2.10 and 2.11.

Campus SSO with SAML

The ckanext-saml2auth extension lets CKAN accept logins from the institution's identity provider, so researchers sign in with their campus account. This reduces local accounts for staff and students and lets access follow group membership from the directory. Confirm with your identity team that the extension supports your CKAN version, how attributes map to roles, and how accounts are removed when someone leaves the university.

Moodle, Open edX and Canvas for teaching

Moodle (GPL-3.0), Open edX (AGPL-3.0) and Canvas LMS (AGPL-3.0) manage courses, enrolments and assessments. They are not dataset publication systems, so they fit a research data catalog only when course datasets must be linked out, for example a teaching dataset cited from a module page. In that case, the catalog should host the data and the LMS should hold a link, not a copy.

Strapi, Payload or Directus for a project website

A headless CMS can run a separate project or institute website that describes datasets and links into the catalog. Strapi is MIT licensed except its ee directories, which form the separately licensed Enterprise Edition. Payload is MIT licensed. Directus uses the Monospace Sustainable Core License 1.0, which allows internal use and professional services. Keep the website separate from the catalog so that content editing does not change dataset records.

How a university rollout moves from data inventory to supported catalog

The sequence turns a set of departmental datasets into a governed CKAN research catalog with agreed metadata, access tiers, campus sign-in and tested behavior before anyone outside the project depends on it. Each phase closes with a written artifact that the next phase relies on.

  1. Inventory and classify datasets by department

    Research offices, labs and institutes list the datasets they hold, where those datasets live, who created them and what personal or sensitive content they contain. The output is a classification register that records each dataset's owner, sensitivity, funder obligations and whether it may be published, shared under restriction or kept internal. The step is done when every department has signed off its entries and the data steward has reviewed the register for gaps.

  2. Agree the metadata profile and identifier decisions

    The project team, library staff and data stewards choose the required metadata fields, controlled vocabularies and how datasets will carry persistent identifiers and citation details. Where funders ask for specific data sharing statements or repository details, those fields are added to the profile at this stage. The output is an approved metadata profile with field definitions, mandatory and optional fields and examples, and a written decision on the identifier scheme and the citation format.

  3. Design access tiers and obtain sign-off

    Open, restricted and internal datasets are separated into access tiers, and each tier states who may view metadata, who may download files and who may query the DataStore. Where human-subjects approval, data use agreements or a data steward governs a dataset, the approving body confirms the tier in writing before publication. The output is an access-tier design document and a signed approval record for each restricted dataset, which the test plan later checks.

  4. Map campus identity to portal roles

    The team connects the portal to the university identity provider, typically through SAML 2.0 when the campus uses it, and maps directory groups such as faculty, graduate students, project staff and external collaborators to CKAN organizations and roles. Confirm with the identity team which SSO extension and version your CKAN release supports before committing to it, since the ckanext-saml2auth extension is a third-party project with its own maintenance status. The output is a role matrix and a tested sign-in and sign-out flow for each group.

  5. Build and test on a supported CKAN release

    Engineers install CKAN from source or from a managed container image on a release in the supported line, with PostgreSQL, Solr and Redis configured as the installation guide describes, and configure the metadata profile, DCAT output if needed and the DataStore with separate write and read-only database users. Staging runs the access tiers, SSO mapping and ingestion of sample files through XLoader. The output is a staging environment, a configuration record and a test report that shows each access rule and each metadata rule working as designed.

  6. Migrate pilot datasets and verify records

    A small set of pilot datasets, chosen from different departments and sensitivity levels, is moved into staging with its metadata, resource files and access tier. Stewards check that titles, identifiers, licenses, contact details and restrictions match the register and that DataStore imports produce the expected row counts. The output is a migration log and a signed verification list per pilot dataset, which becomes the template for the remaining migration.

  7. Test accessibility and security before launch

    Testers check the public interface against WCAG 2.1 Level AA where the institution is subject to accessibility obligations, review access control and authorization, test the upload and API surfaces and confirm that logs record access and disclosures for regulated records. Findings are triaged with owners and fixed or formally accepted. The output is a test report with severity ratings, a remediation list and a retest result, which the security owner signs before go-live.

  8. Go live and support the catalog

    The catalog moves to production with the approved configuration, the monitoring and backup routines and a named support contact for data stewards and researchers. After launch, patch releases are applied on the supported schedule, metadata profile changes go through the same approval path and periodic access reviews confirm that group memberships still match campus roles. The output is a go-live record, an operations runbook and a support agreement that defines response paths for access and data quality issues.

Which integrations and hosting choices change the university build

How does campus SAML single sign-on change a CKAN research data portal?

Campus SAML single sign-on lets researchers and staff log in with their existing university account, and it lets the portal map campus group membership to CKAN roles and organizations. The community extension ckanext-saml2auth is one option to evaluate, but its fit with your identity provider, attribute release rules and CKAN version must be tested rather than assumed. The campus identity team owns the identity provider, the metadata exchange and attribute release policy. DeployOpen can plan and implement the integration as part of a fixed-scope engagement. Confirm with campus IT which attributes they release, whether they support SAML 2.0 for your service, how they handle leavers and whether local accounts remain for external collaborators.

What do the DataStore and XLoader add to a research dataset catalog?

The DataStore holds structured data from resources in a separate PostgreSQL database and exposes a Data API for filtering, full-text search, pagination and SQL queries. XLoader is the officially supported tool for loading uploaded or linked files into it, while DataPusher is now legacy. This matters for research data because tabular files become queryable without downloading them. Your engineers own the database sizing, the write and read-only database users and the loading jobs. Confirm with your research data team which file formats they need loaded, how large the largest files are, and whether access restrictions on a dataset must also apply to its DataStore tables.

Why do DCAT RDF endpoints matter for university data harvesting?

DCAT RDF endpoints publish dataset metadata in a standard vocabulary that other catalogs and aggregators can harvest. The ckanext-dcat extension provides DCAT-AP v1, v2 and v3 schemas, RDF endpoints and an RDF harvester, and it is supported on CKAN 2.10 and 2.11. Which profile you need depends on the aggregators and funders you report to, so confirm that before building. Your metadata librarians own the field mapping and your platform team owns the endpoints. Ask the library whether a national or disciplinary aggregator requires a specific profile, and confirm that only datasets marked as open appear in the RDF output.

Should a university host the research data portal on campus or in research computing?

The choice depends on where the controlled data already lives, who runs the servers and what your security office accepts. A campus data center, a research computing cluster or a cloud account under university control can all host a self-hosted CKAN stack, which needs Python 3.10 or newer, PostgreSQL 12 or newer, Solr 9 or 8 and Redis. Hosting in the institution's own environment keeps records under its direct control, which supports the FERPA contractor conditions. Confirm with the CIO and research computing lead who owns patching, backups, network segmentation and the on-call rota, and what they will pay for the infrastructure.

How should backup and restore work for controlled research datasets?

Backups must cover the CKAN database, the DataStore database, the file storage for uploaded resources and the search index configuration, and restores must be tested with the same care as backups. Controlled datasets may need separate retention, encryption and access rules from open ones, so the backup scope and keys should follow those classes. Your platform team owns the schedule and restore tests, and your data governance office owns retention rules. Confirm the recovery point and recovery time targets with the principal investigators and sponsors, check that restored access restrictions still match the original, and ask who can restore data and how that action is logged.

What logging supports FERPA access records in a research portal?

FERPA requires institutions to keep a record of each request for access to and each disclosure of personally identifiable information from education records, including the parties and their legitimate interests, for as long as the records are kept (34 CFR 99.32). A portal that holds student-related data needs audit logs that capture who viewed or downloaded a restricted resource, when, and under which role, and those logs must be protected from alteration. Your compliance office defines what counts as a disclosure, and your platform team configures and retains the logs. Confirm with the registrar and general counsel whether any dataset contains education records at all, and whether the logs must be exported to the campus security monitoring system.

Who owns the integrations once the research data portal goes live?

Ownership usually splits between the campus identity team, the research computing or platform team, the library or research data office and the implementation partner for the fixed-scope build. Each integration has a different owner, and unclear handoffs are a common source of delay. DeployOpen plans, implements, integrates, documents and supports self-hosted platforms inside your environment, and ongoing support and upgrades are available as a retainer. Ask any partner for a written responsibility matrix covering identity, database operations, harvesting endpoints, backups and log retention before work begins.

What auditors and compliance offices ask to see from the portal

This table lists the evidence a compliance office or external assessor typically requests, the portal component that produces each item, and what the reader should confirm before the assessment. DeployOpen prepares this evidence, but auditors, regulators and the institution's own compliance office assess it and decide whether it is sufficient.

Evidence itemProduced byWhat to confirm before the assessment
Access logs for controlled datasetsCKAN activity stream and reverse proxy logs, forwarded to the campus log storeLogs record who viewed or downloaded restricted datasets, and retention matches the institution's record-keeping policy
Role and organization configuration exportCKAN organizations, roles and dataset permissions, exported on a scheduleEach school official's access matches a documented legitimate educational interest
SSO and MFA configurationCampus identity provider and the ckanext-saml2auth connector, or the institution's chosen SSO methodMulti-factor authentication applies to every account that reaches a system holding student or sensitive data
Written information security programInstitution policy set, with the portal's security controls mapped into itA designated coordinator is named and the program covers the portal, its hosting and its backups
Retention and deletion rulesDataset lifecycle settings and documented deletion procedures in the portal runbookRules for personal data in deposited datasets are agreed with the data protection office before the portal goes live
Disclosure and request recordAccess request workflow and logs for each release of personally identifiable informationRecords are kept for as long as the underlying records are kept, as the institution's policy requires
DCAT output validation reportckanext-dcat RDF endpoints, validated against the DCAT-AP profile the institution usesThe validator and profile version are recorded with each test run, and the funder's metadata fields are present
Accessibility test reportsScanning and manual test results for the portal's public pages and search interfaceResults are tied to WCAG 2.1 Level AA and the dates of the most recent retest

How persistent identifiers and citation fit a CKAN catalog

CKAN records dataset metadata: title, description, creators, licence, dates and custom fields. It does not, on its own, mint a persistent identifier that resolves to a dataset for the whole research community. A DOI is issued by a registration agency or a repository service that the institution has contracted or joined, and each agency sets its own metadata requirements, fees and prefix rules. Before the catalog design is frozen, confirm with the library or research office which agency or repository will issue DOIs, whether the institution already holds a membership or prefix, and whether that provider accepts a metadata push from CKAN or expects a manual deposit. Those answers decide whether the portal stores the identifier as a field, generates it through an integration, or only displays a reference to an external record.

Citation depends on fields that the dataset schema must hold from the start. Creators and contributors should carry persistent researcher identifiers, the publisher should be the institution, the publication year should be fixed at release, and a citation string should be generated from those fields rather than typed by hand. Versioning needs a decision too. When a dataset changes after a DOI has been issued, most teams keep the original record citable and publish a new version with its own identifier, linked to the earlier one. In CKAN, that usually means a dated version field, a consistent naming pattern for resources, and a clear note on each change. Test the process with an illustrative dataset that is revised twice before you commit to a versioning policy.

Funder requirements reach the catalog through the same metadata. Funders often ask for data availability statements, embargo end dates, licence terms, funding award numbers and the location of the data. Add these as explicit fields in the CKAN schema, then map them to DCAT properties with ckanext-dcat so that the RDF output and any harvester downstream carry the same values. Check each funder policy against the fields you plan to expose, and confirm the mapping with the people who answer funder queries. Once the schema is in place, review its output with a sample record rather than assuming the extension carries custom fields through unchanged.

What does a university research data portal cost to build and run?

What drives the cost of a university research data portal?

The biggest drivers are the volume and variety of datasets, the number of access tiers, the complexity of campus single sign-on, the amount of legacy data to migrate, and how many metadata profiles the portal must support. Each one adds design, configuration and testing work, so the written scope should list them explicitly. No price is given here because quotes depend on the written scope. Ask any provider to break its quote into requirements analysis, installation, integration, migration, testing and handover, and to state which items are fixed and which are priced by the day.

Which costs are one-time and which recur after launch?

One-time costs usually include design, installation, integration with campus identity, migration of existing catalogs, metadata mapping and acceptance testing. Recurring costs include hosting and backups, applying CKAN patch releases, planning minor upgrades, monitoring, and support for staff and researchers. CKAN targets about two minor releases a year, and only the latest patch of the current minor version and the previous minor version receive security and bug fixes, so the team must plan for upgrades on a steady cycle. Ask for the recurring items to be priced separately from the build.

Can our own team build a CKAN research data portal without a partner?

A capable team can, but the effort is larger than a demo install suggests. Installing CKAN from source requires Python 3.10 or newer, PostgreSQL 12 or newer, Solr 9 (recommended) or Solr 8, and Redis, with uWSGI and Nginx recommended for production. The team must also set up the DataStore with separate write and read-only database users, configure a search index, add single sign-on, and keep up with patch and minor releases. Check whether your staff have these skills, who will be on call, and whether the work fits alongside existing duties before deciding.

How long does a research data portal take, and what must the institution provide?

Timelines depend on scope, the number of datasets to migrate and the speed of institutional approvals, so no duration is given here. The institution must provide data stewards who can classify datasets as open or restricted, IT staff to approve hosting, network and identity integration, and a named sign-off owner for each phase. Research office and privacy staff should review access rules before any restricted dataset is loaded. Ask the provider to show these dependencies in the plan and to name which milestones need institutional sign-off.

How does campus single sign-on affect the cost of the portal?

Single sign-on adds effort in proportion to how many identity sources, roles and attributes the portal must map. Connecting a campus SAML identity provider is different work from building local accounts, and it requires testing with real student, staff and external researcher accounts. The ckanext-saml2auth extension is one community option to confirm against your identity provider and CKAN version. Ask the provider to list the identity protocols, attribute mappings and test cases included in the fixed scope.

What does migration of existing datasets and metadata add to the budget?

Migration cost depends on how many datasets exist, how consistent their metadata is, and whether data files need to be loaded into the DataStore. Files loaded into the DataStore need the XLoader tool, which is the officially supported loader, since DataPusher is now legacy. Cleaning metadata, mapping old fields to a standard profile, and validating the results usually take more time than the technical import. Ask for a sample migration with a reconciliation report before committing to the full migration.

Which compliance evidence should we ask the provider to include in the scope?

Ask for evidence that supports your obligations, such as access logs for restricted datasets, documented role-based access tiers, backup and restore records, and the data retention and deletion procedure. Depending on the data, FERPA, GLBA Safeguards Rule and COPPA may apply, and each sets different expectations for access records and security programs. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports or certifications. Confirm the required evidence list with your compliance office before the scope is signed.

How DeployOpen scopes and supports a research data portal

Every engagement starts with a written scope. For a research data portal, that scope should name the CKAN version and extensions, the DataStore datasets, the identity provider the campus uses, the access tiers for open and restricted datasets, and the funder metadata fields the portal must carry. DeployOpen agrees the scope, deliverables and timelines in writing before work begins, so a research office can check what is included against its data management plans and its institutional review requirements. Ask for the exclusions as clearly as the inclusions, including who maintains the metadata schema after launch and who approves changes to dataset access rules. DeployOpen does not publish a price list; a quote follows from the written scope.

The portal runs inside the institution's own infrastructure, whether that is a campus data center, a university cloud tenancy or a research computing environment. Confirm early where the DataStore PostgreSQL databases, the Solr search index and the file storage will sit, who holds administrative credentials, and how backups leave the environment. Restricted datasets often carry obligations under FERPA or, for health research, other regimes, so the hosting decision should be made with the data protection office rather than after the portal is built. DeployOpen plans, hardens and documents that environment, and it runs security testing against the portal and its APIs before launch and when major changes ship, then reports findings for the institution to remediate.

DeployOpen prepares the evidence a compliance reviewer will ask for, such as configuration baselines, access control records, logging settings and the test results behind them. It does not issue audit reports, attestations or certifications; those come from the institution's independent auditors or certification bodies. After go-live, ongoing upgrades, patching of CKAN minor and patch releases, and continuous testing can be covered by a retainer, which keeps a named support path open for the research data office. Ask any partner to state in writing which CKAN releases it will track, how it handles extension updates, and what response it commits to for security issues, and confirm those terms before signing.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.