v2.0.0_RC4

What's Changed

• Removed the LIMIT 1 condition from the nested queries of domain and question chart by @prashanthShiksha in #100
• Fix [3262] : User bulk uploaded with program manager role was not able to login to metabase by @Vivek-M-08 in #101

Full Changelog: v2.0.0_RC3...v2.0.0_RC4

v2.0.0_RC3

What's Changed

• Fix[3097] : Big number count for Projects "No. of improvements in inProgress status currently" is mismatching by @prashanthShiksha in #88
• updated the logic to download the maven in the Dockerfile by @prashanthShiksha in #89
• Data-pipeline Fix[3087] : Same user who is consuming Obs from 2 different schools, clusters or states. Only 1 record has been tracked in DB & same is reflected in dashboard. by @Vivek-M-08 in #90
• Fix [3110] : Survey reports are not getting generated in the Metabase dashboard. by @prashanthShiksha in #92
• Added the create tables logic outside by @Vivek-M-08 in #93
• Fix [3087] Same user who is consuming Obs from 2 different schools, clusters or states. Only 1 record has been tracked in DB & same is reflected in dashboard. by @prashanthShiksha in #91
• added the new column config by @prashanthShiksha in #94
• Migration Script Enhancements by @Vivek-M-08 in #95
• Modified .dockerignore file by @Vivek-M-08 in #96
• Refactor: Extract parentInfo data logic with helper methods and entity type optimization by @Vivek-M-08 in #98
• Domain dashboard query changes by @prashanthShiksha in #97
• added the logic to update the domain dashboard according to entity by @prashanthShiksha in #99

Full Changelog: v2.0.0_RC2...v2.0.0_RC3

v2.0.0_RC2

What's Changed

• User service changes and Bug fix for Id 3039 by @Vivek-M-08 in #86

Full Changelog: v2.0.0_RC1...v2.0.0_RC2

v2.0.0_RC1

What's Changed

• User Service Integration with Metabase by @Vivek-M-08 in #77
• Updated the question data storing logic + merged the domain and question dashboard + stored the school udise code + certificate status by @prashanthShiksha in #79
• resolved the automatic flink job stopping issue by @prashanthShiksha in #81
• Create groups right after collection is created by @Vivek-M-08 in #82
• Database cleanup by @Vivek-M-08 in #84
• Fixed NullPointerException issue while pushing notifications by @Vivek-M-08 in #85

Full Changelog: v1.0.0...v2.0.0_RC1

v2.0.0

🚀 Release: Real-Time User Service Integration with Metabase

📌 Overview

This release brings the real-time integration of the User Service with Metabase, enabling automatic user creation, role-based access provisioning, and dashboard visibility—all powered through Kafka event streams. Additionally, we've made important updates to certificate tracking, school UDISE code logic, and dashboard structure.

---

✅ Finalized Decisions

• Metabase Login Support:
  Users are created using username@domain.com static domain name, followed by random Passwords. If users have registered using the phone.

• Role Update Handling:
  Profile role changes are tracked, and Metabase access is updated automatically.

• Program Manager Mapping:
  Program managers are mapped to the appropriate program based on the user creation event payload.

📄 Design Document
🧪 Test Case Sheet
🔗 Pull Request #77

---

🛠️ Features & Implementation

⚙️ Flink Job Setup

• A new Flink job listens to Kafka events for user lifecycle changes:
  • create – Creates Metabase user with appropriate access.
  • update – Adjusts roles and dashboard visibility.
  • delete – Revokes access and deactivates user.

👥 User Access Logic

• New Users: Automatically created with assigned roles and dashboard access.
• Updated Users: Access adjusted in Metabase collections based on updated roles.
• Deleted Users: Fully removed from Metabase collections.
• Program Managers: Mapped to specific programs per event data.

🔔 Notifications

• Sends onboarding email and mobile notifications after user provisioning.

---

📘 Enhancements

🏫 School UDISE Code Handling

• Improved mechanism for managing and tracking UDISE code updates per user or institution.

🏆 Certificate Tracking Enhancements

Stored in project data under certificate object:

• 'pdfPath' present → Issued
• 'certificate.eligible = true' → Eligible
• 'certificate.eligible = false' → Not Eligible
• 'transactionId present & pdfPath missing' → In Progress

🏢 Tenant & Role Metadata

The following fields are now stored:

• tenant_code, org_code, org_name
• roles, roleIds

---

📊 Dashboard Improvements

🔁 Merged Dashboards

• Observation dashboards with Rubric - Domain and Rubric - Question have been merged for a unified reporting experience.

---

🧪 Testing & Validation

• All functionalities tested against real Kafka events.
• Test results and edge case handling are documented in the Test Case Sheet.

---

SaaS QA fixes

• fix[2873] : [R1.0.0 SAAS_QA_MD] In the dashboard, the counts for 'In Progress' and 'Started' status are not reflecting for both Observation and Survey, and in-progress observations were coming as draft in the status report.
  Ticket : BUG2873
  Resolution:
  Added status-checking conditions in the Project, Survey, and Observation stream functions.
  Now, the system will process only the following status events: started, inprogress, submitted, and completed.
  All other statuses will be ignored.

• fix[2922] : [R1.0.0 SAAS_QA_MD] In survey question report, Evidence field is missing.
  Ticket : BUG2922
  Resolution: Added the evidence column.

What's Changed

• Project structure setup and solution level data extraction by @Vivek-M-08 in #1
• project level data extraction by @Vivek-M-08 in #2
• added PostgresUtil file by @Vivek-M-08 in #3
• task level data extraction by @Vivek-M-08 in #4
• fix for local flink testing by @Vivek-M-08 in #5
• fix for elevate events by @Vivek-M-08 in #6
• Dev deploy by @Vivek-M-08 in #7
• changes in org and location data by @Vivek-M-08 in #8
• adding scripts for Improvement Projects Dashboard by @Vivek-M-08 in #9
• Added filter dropdown by @Vivek-M-08 in #10
• DataPipeline: Certificate changes by @Vivek-M-08 in #12
• Dev deploy by @prashanthShiksha in #11
• Updated database name by @Vivek-M-08 in #13
• Added a docker compose file for metabase by @Vivek-M-08 in #14
• Metabase: Updated docker compose file by @Vivek-M-08 in #15
• modified the run.sh and file path variables by @prashanthShiksha in #16
• Added the program-report, state-report and super-admin-report related script by @prashanthShiksha in #17
• Metabase - user management for metabase by @Vivek-M-08 in #18
• Flink-Job: Separated modules by @Vivek-M-08 in #19
• Flink Job: Changed the directory name by @Vivek-M-08 in #20
• Added project dashboard creation files by @prashanthShiksha in #21
• Flink Job: [1380] Publish Dashboard Details to Kafka on Unique Submission by @Vivek-M-08 in #22
• Flink Job: Add Metabase API Integrations to Core-Jobs Module for Dashboard Automation by @Vivek-M-08 in #23
• Flink Job: Migrated project-dashboard-creator module to metabase-jobs module by @Vivek-M-08 in #24
• Flink Job: Fix to get location data for dashboard kafka event by @Vivek-M-08 in #25
• Flink Job: Corrected the kafka message by @Vivek-M-08 in #26
• Akka API: [1432] Implement CSV Upload API for Metabase User Management by @Vivek-M-08 in #27
• Metabase API: Added methods in MetabaseUtil file to create groups by @Vivek-M-08 in #30
• [data-pipeline] Code enhancement by @Vivek-M-08 in #31
• [data-pipeline] Code enhancement by @Vivek-M-08 in #32
• Added the Project Dashboard Creation Files with the scala by @prashanthShiksha in #29
• Metabase script integrated with postgres by @prashanthShiksha in #33
• Added 2 new columns in solutions table by @Vivek-M-08 in #34
• Code clean up and submit job command changes by @Vivek-M-08 in #35
• Fix to NullPointerException for empty userRoles by @Vivek-M-08 in #36
• Qa drop 1 (demo action items addressed) by @prashanthShiksha in #37
• resolved cluster param issue by @prashanthShiksha in #38
• Code Fix : Error inserting data Unterminated string literal by @Vivek-M-08 in #39
• Bug fix to get project objective by @Vivek-M-08 in #40
• Added basic authorization for AKKA API's by @Vivek-M-08 in #41
• Fix: Irrelevant Data in Filters and Duplicate Submissions in Status Reports by @prashanthShiksha in #42
• changed from local_projects to config.projects by @prashanthShiksha in #43
• fix: Resolve Duplicate Program Issues Across States by @prashanthShiksha in #44
• Flink Job: Mi dashboard by @Vivek-M-08 in #47
• Revert "Flink Job: Mi dashboard " by @Vivek-M-08 in #48
• Dockerized setup of elevate report service by @prashanthShiksha in #46
• First release to main branch by @Vivek-M-08 in #49
• added the contributors in the readme file by @prashanthShiksha in #50
• Release 1.0.0 by @Vivek-M-08 in #51
• Revert "Revert "Flink Job: Mi dashboard "" by @Vivek-M-08 in #52
• added the user_data.csv file path by @prashanthShiksha in #53
• Release 1.0.0 by @Vivek-M-08 in #54
• Flink Job : Mi dashboard changes by @Vivek-M-08 in #55
• Auto submit the jar file when container restarts by @prashanthShiksha in #56
• Revert "Flink Job : Mi dashboard changes" by @Vivek-M-08 in #57
• changed the the branch name in the download path by @prashanthShiksha in #59
• Release 1.0.0 [21/04/25] by @Vivek-M-08 in #60
• changed the elevate-data image version by @prashanthShiksha in #61
• Release 1.0.0 [21/04/25 - 02] by @Vivek-M-08 in #62
• Moved the flink jobs into a structured modules by @Vivek-M-08 in #63
• Survey and Observation jobs are added by @Vivek-M-08 in #65
• Added the metabase and stream job jar file paths by @prashanthShiksha in #66
• updated the evidence url by @prashanthShiksha in #68
• added the criteria param by @prashanthShiksha in #69
• Resolved the obs-without-rubric heading issue & modified slider and text chart for survey and obs by @prashanthShiksha in #70
• Updated the task-report query by @prashanthShiksha in #71
• Bug fixes fro the following tickets by @Vivek-M-08 in #72
• fix[2873,2922] added the status checking logic by @prashanthShiksha in #76

New Contributors

• @prashanthShiksha made their first contribution in #11

Full Changelog: https://github.com/ELEVATE-Project/data-pipeline/commits/v1.0.0