Tender Management API - Fetching Tenders by Organization with User-Specific Filters

1. Key Features:

• Pagination and Filtering: The API allows pagination (page, limit) and filtering by searchName, tenderStatus, and organization_id to retrieve relevant tender data.
• User-Specific Access Control: It incorporates user role-based filtering, allowing different roles (e.g., procurement_manager, procurement_officer, reviewers) to access tenders based on their permissions.
• Sorting and Grouping: It fetches tenders sorted by unique_tender_id and created_at in descending order, ensuring that the latest tenders are grouped and returned.
• Reviewer Data Embedding: Embeds detailed information about reviewers for each tender, including their email and role.
• Process and Template Details: Each tender is enriched with details from related process_id and template_id data.
• Search Functionality: Supports searching by template_data.name and returns the filtered results.

2. Key Functionalities:

• Data Fetching:

  • The route uses query parameters (page, limit, searchName, tenderStatus, etc.) to retrieve filtered tender data from MongoDB.
  • It performs aggregation queries to handle sorting, grouping, and pagination. The use of aggregate allows for complex data transformations like sorting by unique_tender_id and created_at, grouping by unique_tender_id, and counting the total tenders after the filters are applied.

• User Role Filtering:

  • The user’s role (user_type) is retrieved based on user_id. This determines how the query will filter the tenders, e.g., a procurement_manager can access tenders they are reviewing, while a procurement_officer can only access tenders they created.
  • The filtering logic involves conditions like checking if the user is a reviewer in various stages (reviewers_TOR, reviewers_EOI, etc.) or if the user is the creator of the tender.

• Pagination:

  • The skip parameter is used to calculate how many tenders to skip based on the page number and limit, ensuring that only a subset of tenders is returned in each request.
  • The limit is set to 5 tenders per page, which is hardcoded but could be adjusted to suit the requirements.

• Reviewer Data Embedding:

  • The code extracts all reviewer IDs for the fetched tenders, queries the user_collection to fetch reviewer details, and then embeds these details (e.g., email, role) into each tender.
  • This ensures that each tender has enriched reviewer information without needing additional database queries for each individual reviewer.

• Template and Process Details:

  • The tender’s process_id is used to fetch related process details (if available). Similarly, template IDs (template_id_tor, template_id_eoi) are resolved and embedded into the tender document for comprehensive data representation.

• Search Filtering:

  • If a searchName query parameter is provided, the tenders are filtered based on a case-insensitive regex match to the template_data.name, ensuring that only tenders whose templates contain the search term are included in the response.

3. Data Fetching and Step Handling:

Data Fetching:

• MongoDB Aggregation Pipeline:
  • The aggregate pipeline in MongoDB is used to perform complex operations like sorting, grouping, and limiting the returned data. The key stages of the aggregation are:
    • $match: Filters tenders based on query criteria (like tenderStatus, organization_id, etc.).
    • $sort: Sorts the tenders first by unique_tender_id (ascending) and then by created_at (descending).
    • $group: Groups the tenders by unique_tender_id, ensuring that only the latest tender per group is returned.
    • $replaceRoot: Replaces the root document with the latestTender from the previous aggregation stage.
    • $skip and $limit: Implement pagination by skipping a certain number of records and limiting the results to a maximum count.

Step Handling:

• Pagination: The page and limit parameters allow for pagination, with the skip calculated based on the current page and limit. This enables the API to return a subset of tenders at a time, which is crucial for efficient data retrieval in large datasets.

• User-Specific Filters: Based on the role of the user (if user_id is provided), the query is adjusted to either show all tenders a user is involved with (e.g., procurement_manager role) or restrict tenders to those created by the user (e.g., procurement_officer role).

• Data Enrichment:

  • Reviewer IDs: After retrieving the tenders, the code gathers all reviewer IDs across different stages and fetches their details from the user_collection in bulk, reducing the number of database queries.
  • Process and Template Information: For each tender, the process_id and template IDs are resolved and added to the tender object to enrich the returned data.

Example Code for Reference:

# Example of data fetching and processing using MongoDB aggregation
latest_tenders = list(tenders_collection.aggregate([
    {"$match": query},  # Step 1: Apply user-specific query filters
    {"$sort": {"unique_tender_id": 1, "created_at": -1}},  # Step 2: Sort by tender ID and creation date
    {"$group": {"_id": "$unique_tender_id", "latestTender": {"$first": "$$ROOT"}}},  # Step 3: Group by tender ID
    {"$replaceRoot": {"newRoot": "$latestTender"}},  # Step 4: Replace root with the latest tender
    {"$skip": skip},  # Step 5: Implement pagination
    {"$limit": limit}  # Step 6: Limit results to the specified number
]))

This code fetches tenders by applying multiple filters (status, organization, etc.), enriches them with reviewer and process data, and supports pagination.