Indira Gandhi Institute of Development Research Students: Serving Economic Datasets Through a Python API

Yes, an economics student can build a clean, useful API for a public dataset in 14 days using Python, a small web framework and a lightweight database. The key is to keep the scope narrow: one dataset, three or four endpoints, honest data cleaning, and a citation on every response so the numbers can be traced back to their source.

This guide gives IGIDR students a day-by-day plan, a data-cleaning checklist, a source-citation template, and an illustrative design for filtered and paginated endpoints. The tools are Python, FastAPI, pandas and SQLite, but the same plan works with Flask or Django REST Framework.

Why an economics student should serve data through an API

Most economics work starts with a spreadsheet that someone downloaded months ago. An API changes this in three ways:

• Reproducibility: a script or notebook can request the exact slice it needs (for example, one state, one indicator, 2005 to 2020) instead of relying on a manually edited file.
• Documentation of decisions: every cleaning rule you apply must be written as code, so your treatment of missing values and unit conversions becomes visible and reviewable.
• Portfolio value: a small, well-documented API shows backend skills (routing, validation, pagination, testing) applied to a domain you actually understand.

Scope of the finished API: endpoints, filters and limits

Decide the contract before writing code. A modest contract is easier to finish and test.

A 14-day project plan for the dataset API

The plan assumes roughly two to three focused hours a day. Adjust the pace if you are also handling coursework.

How to choose a public economic dataset that suits a two-week build

The wrong dataset can consume all 14 days in cleaning alone. Screen candidates against these points:

• Tabular and modest in size: tens of thousands to a few hundred thousand rows is plenty. Avoid sources that need scraping from PDFs.
• Clear metadata: units, definitions, base year and frequency should be documented by the publisher.
• Stable identifiers: country, state or sector codes help joins and filters.
• Explicit licence or terms of use: read the publisher’s terms before redistributing data through an API, and note them in your repository.
• Interesting enough to explain: pick something you can discuss in an interview, such as state-level indicators, price indices or trade series.

Government open-data portals, central-bank statistical databases and multilateral agency data portals are the usual places to look. Whichever you use, take the definitions from the publisher’s own documentation rather than from memory.

Data-cleaning checklist for economic time series and panel data

Use this as a literal checklist in your notebook. Tick each item and log what you changed and how many rows it affected.

Two principles matter most for economics data: never invent a value to fill a gap, and never delete a strange value without writing down why. An API consumer who sees a null with a status flag can make an informed choice; one who receives a silent zero cannot.

Source-citation template for every dataset and API response

Keep the citation as structured fields in the database, then render it as text in the README and in API responses. Copy this template into SOURCES.md and fill one block per source:

Dataset title:
Publisher / organisation:
Series or table name:
Original URL (exact page you downloaded from):
Date retrieved (DD Month YYYY):
Publisher's last-updated date (if shown):
Version / edition / release:
Licence or terms of use (link + short summary):
Coverage (regions, period, frequency):
Units and price basis:
Transformations applied by this project (link to CLEANING_LOG.md):
Suggested citation (one line):

A suggested one-line citation could follow this pattern, with your own details in place of the brackets:

[Publisher]. ([Year of release]). [Dataset title], [series name]. Retrieved [date] from [URL]. Processed by [your name] for [project name]; see CLEANING_LOG.md for changes.

In the API, include a source_id on each observation and let /sources return the full block. This keeps responses light while keeping every number traceable.

Designing filtered and paginated endpoints

Filters

• Validate every parameter. Reject year_from greater than year_to with a clear message.
• Use exact matches for codes (region=MH) and keep any free-text search on a separate parameter.
• Always use parameterised queries, never string concatenation, to avoid SQL injection.
• Index the columns you filter on: indicator, region and period.

Pagination

• Use limit and offset for a first version. Set a default (say 50) and a maximum (say 500).
• Always apply a deterministic sort, such as period then region then id. Without it, pages can overlap or skip rows.
• Return the total count for the filtered query so clients know when to stop.
• If time remains, note in the README that cursor-based pagination is better for data that changes frequently.

Worked example: one filtered request, start to finish

This is an illustrative design sketch with made-up values, not output from a run. Imagine a cleaned table observations with columns id, indicator_code, region_code, period, period_type, value, unit, status, source_id.

A client asks for per-capita income-style series for one state between two fiscal years, five rows per page:

GET /observations?indicator=pc_income&region=MH&year_from=2015&year_to=2020&limit=5&offset=0&sort=period

A sketch of the handler logic:

@app.get("/observations")
def list_observations(indicator: str, region: str | None = None,
                      year_from: int | None = None, year_to: int | None = None,
                      limit: int = Query(50, ge=1, le=500),
                      offset: int = Query(0, ge=0)):
    if year_from and year_to and year_from > year_to:
        raise HTTPException(422, "year_from must be <= year_to")
    where, params = ["indicator_code = ?"], [indicator]
    if region:
        where.append("region_code = ?"); params.append(region)
    if year_from:
        where.append("period_start_year >= ?"); params.append(year_from)
    if year_to:
        where.append("period_start_year <= ?"); params.append(year_to)
    clause = " AND ".join(where)
    total = db.execute(f"SELECT COUNT(*) FROM observations WHERE {clause}", params).fetchone()[0]
    rows = db.execute(f"SELECT * FROM observations WHERE {clause} "
                      "ORDER BY period_start_year, region_code, id LIMIT ? OFFSET ?",
                      params + [limit, offset]).fetchall()
    return {"count": total, "limit": limit, "offset": offset, "results": rows}

Design points this example makes visible: the f-string only inserts fixed clause text, never user input; the sort is deterministic; the year filter uses a numeric period_start_year derived from the fiscal label (so “2019-20” becomes 2019), a decision that must be written in the cleaning log. Then test the edges: offset beyond the count should return an empty list, not an error, and limit=0 should return 422.

IGIDR students: fitting an API project around an economics research programme

A third-party admissions aggregator, which is not an official source, describes IGIDR as an autonomous research institute set up by the Reserve Bank of India in 1987 and made a Deemed-to-be University in 1995. The same page lists an MSc Economics, an MPhil-PhD Integrated Programme in Development Studies and PhD programmes, and places the campus in Goregaon East, Mumbai. Treat these as excerpt-level details from 2026 and check them against the institute’s own website.

What this means for this project: the programme mix points towards quantitative economics and development research, where cleaning and citing data carefully is part of the daily work. A dataset API is a natural companion to that: it can back a replication exercise, a term-paper data pipeline or a small research tool. At the same time, it is not safe to assume that your department has a formal internship, industrial-training or project-credit rule, or that a Python API project can count towards coursework or a thesis. Those rules are not something this guide can state for you.

Practical advice: pick a dataset related to something you already study, such as regional development indicators, so the project strengthens your academic work instead of competing with it. Keep your documentation strong enough that a faculty member could review the cleaning decisions in ten minutes.

Verification steps for IGIDR students before you start

1. Open the official IGIDR website and confirm the programme names, campus details and founding information listed above.
2. Ask your programme coordinator or academic office in writing whether an external, self-directed online internship is permitted during your semester or break, and whether any form of intimation is required.
3. Check the academic calendar for your batch to find a 14-day window that does not clash with coursework, exams or fieldwork.
4. If you want the work to count towards credit, a dissertation or a research assistantship, ask your supervisor or the academic office for the current written rule instead of assuming it.
5. Check whether the dataset you choose comes from a source that your supervisor recommends, and confirm any data-sharing terms if you plan to publish the API publicly.

Testing and documentation checklist before you call the API finished

• A test for each filter on its own and in combination.
• Pagination tests: first page, last page, page past the end, maximum limit exceeded.
• A test that confirms nulls are returned as null, not zero.
• A test that every observation has a valid source_id.
• README with install steps, how to rebuild the database from raw files, three example requests and a limitations section (revisions, aggregates, fiscal-year handling).
• A .gitignore that excludes virtual environments and large generated files, plus a note on how to re-download raw data.

Using a structured online internship to keep the 14 days on track

If you want deadlines and a structure around the build, the free two-week online internship from CareerSwami runs for 14 days across 26 tracks. It comes with a free offer letter and a free completion certificate after successful completion. Choose a backend-oriented track and use this plan as your project. Whatever structure you use, the repository, cleaning log and citations are what show your work.

Frequently Asked Questions

Which framework is best for a first dataset API: Flask or FastAPI?

FastAPI is a good default because it validates query parameters from type hints and generates interactive documentation automatically, which saves time on days 6 to 10. Flask works equally well if you already know it. The design decisions in this guide apply to both.

Do I need a database, or can I serve the CSV directly with pandas?

For a few thousand rows, pandas in memory is fine. SQLite is still recommended because it forces you to define a schema and a unique key, and it makes filtering and pagination queries simple and fast. It is a single file and needs no server.

How should I handle fiscal years like 2019-20 in filters?

Store the original label as period and add a numeric period_start_year for filtering. Document that “2019-20” maps to 2019. Do not silently merge fiscal and calendar series, and expose period_type so users can tell them apart.

What page size limits are sensible?

A default of 50 and a hard maximum of 500 is a reasonable starting point. The exact numbers matter less than having a maximum at all, so one request cannot return the whole table.

Can I publish my API publicly with someone else’s data?

That depends on the licence or terms of the data publisher. Read them, record them in SOURCES.md, and if the terms are unclear, keep the project to local use or contact the publisher. Publishing the code without the data is a safer alternative.

Should I fill missing values with averages or interpolation?

Not in the base API. Keep missing values as nulls with the publisher’s flag. If you want interpolated series, offer them as a clearly labelled separate field or endpoint and document the method.

Will this project count as an internship at my institute?

That cannot be assumed. Whether a self-directed project or an external online internship is recognised, and whether it carries any credit, is decided by your institute. Ask your academic office for the current written rule before relying on it.

How do I show this project to a faculty member or recruiter?

Share the repository link with a README that opens with a three-line summary, a sample request and response, and a link to the cleaning log. Being able to explain one cleaning decision and one pagination decision in detail is more persuasive than a long feature list.

Related Python & Backend Engineering guides

• Vidhyadeep University Students: Reading Database Query Plans in Your Backend Internship
• Sri Balaji University Students: Backend Internship Project Management With a Simple Kanban Board
• Bengaluru North University Students: Backend Internship Interview Day Plan for Bengaluru Startups
• All Python & Backend Engineering internship guides