Rishanki Jain

One of the most impactful design decisions I made on a healthcare project was implementing Medallion Architecture (Bronze → Silver → Gold) in Databricks. At CVS Health, we were ingesting large volumes of pharmacy claims, eligibility, and provider data coming from PBMs like OptumRx in multiple formats (JSON, XML, Parquet). What I actually did: 🔹 Bronze Layer I landed raw claims and eligibility feeds exactly as received from S3, without transformation. This helped us with auditability, reprocessing, and HIPAA traceability when downstream issues occurred. 🔹 Silver Layer Here I applied real business logic — Deduplication of claims Eligibility validation Standardization of member, provider, and plan identifiers PHI masking rules This layer became the trusted operational dataset used across teams. 🔹 Gold Layer I built analytics-ready tables for reporting, SLA monitoring, and downstream dashboards. These were optimized for Redshift/Snowflake consumption and Power BI reporting, with performance tuning and partitioning. Why this worked in production: Reduced reprocessing effort when source data changed Clear separation of concerns between raw, clean, and business data Made debugging claims issues much faster Helped us stay compliant with HIPAA and GDPR This architecture wasn’t theoretical — it directly supported near real-time eligibility checks and helped reduce claim adjudication errors by ~30%. Sometimes good architecture isn’t about fancy tools — it’s about designing data layers that teams can actually trust. #DataEngineering #Databricks #DeltaLake #MedallionArchitecture #HealthcareData #AWS #Snowflake

3

🔍 Ensuring Data Quality at Scale – Why It Matters 🧠 In regulated domains like healthcare and finance, poor data quality is expensive — not just in dollars, but in decisions. At American Express and CVS Health, I designed robust data quality frameworks that proactively catch issues before they impact analytics. Here’s what I implemented: 🔹 Validation Rules – Null checks, type validation, domain value checks 🔹 Automated DQ Tests – Integrated with DBT, PySpark, and Airflow 🔹 Exception Handling – Built retry + alert logic in ETL workflows 🔹 Data Profiling Dashboards – Tracked freshness, completeness, and anomalies 🔹 Logging & Audit Trails – Implemented lineage, versioning, and result logs 🔹 Compliance Checks – Applied masking and policy rules for PHI/PII governance 🔹 Tools: Great Expectations, DBT, custom PySpark validators, Power BI, and MLflow ✅ Result: Increased trust in reporting, faster audits, and fewer escalations — especially when dealing with claims data and financial transactions. Let’s talk data quality — how do you monitor trust in your pipelines? #DataQuality #DQ #DBT #PySpark #Airflow #GreatExpectations #PHI #Compliance #CVSHealth #AmericanExpress #DataEngineering #Lineage #Monitoring #DataTrust

15

Core Belief: Clinical Innovation via Big Data/Privacy Is Your Evolution, Not Risk – Debunking 'Tech Role' Identity Loss I believe big data/privacy trends represent evolution for clinicians, not risk – challenging the myth that tech roles diminish your identity. Reality: 50% burnout in traditional care (AMA 2024), while digital health amplifies expertise. Big data enables 25% readmission reductions (HealthIT 2025), with privacy frameworks (EU AI Act, HIPAA) demanding your insights for ethical implementation. Evolve to $150-$300/hr consulting, guiding value-based systems that scale impact. Global: OECD 70% adoption by 2030, NHS £1B investment, Australia's 50% growth – your clinical core is essential. This belief fueled my pivot: From limited shifts to innovative systems serving thousands, with resilient freedom. Overcome a myth? Let's discuss. #DigitalHealth #BigData #PrivacyInHealthcare #ClinicianEvolution #HealthcareInnovation

8

𝐂𝐨𝐡𝐞𝐫𝐞𝐧𝐜𝐞 𝐖𝐢𝐭𝐡𝐨𝐮𝐭 𝐂𝐫𝐮𝐞𝐥𝐭𝐲⁣ The core concern regulators raise around risk adjustment is reasonable: diagnoses that drive payment should be coherent—anchored to real encounters, treatments, and longitudinal care. When diagnoses appear once and never again, trust in the system erodes. On that premise, regulators are right.⁣ Where the system breaks is 𝐰𝐡𝐨 𝐛𝐞𝐚𝐫𝐬 𝐭𝐡𝐞 𝐛𝐮𝐫𝐝𝐞𝐧 𝐨𝐟 𝐩𝐫𝐨𝐯𝐢𝐧𝐠 𝐜𝐨𝐡𝐞𝐫𝐞𝐧𝐜𝐞.⁣ Clinicians are increasingly judged—years later—as if they should have anticipated persistence, audit logic, and downstream utilization from a single visit. That is an inhuman expectation. Longitudinal coherence is not a visit‑level cognitive task; it is a s𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐩𝐫𝐨𝐩𝐞𝐫𝐭𝐲 𝐨𝐟 𝐩𝐨𝐩𝐮𝐥𝐚𝐭𝐢𝐨𝐧𝐬 𝐨𝐯𝐞𝐫 𝐭𝐢𝐦𝐞. No clinician, acting in good faith, can reliably predict it in the exam room.⁣ Seen this way, defensive documentation is not abuse—it is rational self‑protection in a system that applies hindsight enforcement asymmetrically.⁣ Analytics could help here, but only if roles are cleanly separated. Machines should detect longitudinal patterns and flag incoherence. Clinicians should attest only to what they actually evaluated and addressed today. 𝐓𝐡𝐞 𝐬𝐲𝐬𝐭𝐞𝐦—𝐧𝐨𝐭 𝐭𝐡𝐞 𝐩𝐫𝐨𝐯𝐢𝐝𝐞𝐫—𝐦𝐮𝐬𝐭 𝐚𝐛𝐬𝐨𝐫𝐛 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐮𝐧𝐜𝐞𝐫𝐭𝐚𝐢𝐧𝐭𝐲.⁣ Coherence without cruelty requires better role design, not more documentation or algorithms in the exam room. Accountability and empathy are compatible—but only if we stop asking clinicians to function as surrogate statisticians. #RiskAdjustment #USHealthcare #AuditWithIntegrity #HealthcareCompliance #HealthcareRisk #AuditAndCompliance

18

2 Comments

INTERVIEW NO. 3 Organization: Kaiser Permanente Role: Data Scientist - IV Round 1 - The interviewer asked me in detail about my experience in data science and machine learning. The interview lasted for around 45 mins and most of it was focused discussing my previous projects in AI/ML. In the end, he asked to write 1 Python program in which he gave me a list which had multiple duplicate values and asked me to find the occurence of each number along with its index. Round 2 - The interviewer went through my resume and started asking questions based on my profile. He tested my coding skills in SQL and Machine learning by giving me hypothetical scenarios. He asked me to write the code for an end to end ML pipeline for predicting whether a patient is a cancer patient or not. In technical interviews, you need to think and come up with a reasonable solution to the problem that is assigned in limited time. Make sure to work on your critical thinking and problem solving skills. 💡 Interview Tip: Avoid using percentage-based metrics in your resume (e.g., “Saved X% of time through ABC method”) unless you can confidently justify those numbers during the interview. Thanks for reading! Follow #rajreacts #interviewprep #aiml #interviews #datascience #machinelearning

31

1 Comment

🏥 How We Cut Pipeline Build Time by 65% Using GenAI + AWS in Healthcare Data Engineering In our healthcare data ecosystem, we handle terabytes of clinical and claims data daily — everything from Epic HL7 feeds to pharmacy transactions and patient encounter data. Initially, our data engineering team spent days building and testing ETL workflows. Every schema change or data mapping meant hours of PySpark and SQL rework. The process was accurate, but not scalable — especially as our cloud infrastructure on AWS (S3, Glue, Lambda, and Redshift) kept growing. That’s when we introduced GenAI-assisted data engineering, tightly integrated into our AWS stack. Here’s what that looked like 👇 🔹 Before: Manual ETL creation using AWS Glue and PySpark scripts. Repetitive SQL transformations and schema mapping across HL7/FHIR models. Code reviews + test validation cycles took ~3 days per pipeline. Frequent performance tuning for Redshift queries and data partitioning. 🔹 After (with GenAI + AWS integration): Used GenAI copilots to auto-generate PySpark transformations and Glue jobs. AI suggested S3 bucket partitioning strategies and optimized join logic for Redshift. Generated validation scripts for schema drift detection in Athena. Automated pipeline documentation pushed to AWS Glue Data Catalog. 📊 Impact: Pipeline build time reduced from ~30 hours to 10 hours. Query performance improved by ~40% due to AI-suggested optimizations. Data validation accuracy up 45%. Engineers now focus on business logic and compliance workflows instead of low-level data plumbing. 💡 Why this matters in healthcare: Data accuracy, traceability, and compliance are everything. By using GenAI inside AWS-native tools, we’re accelerating development without compromising HIPAA or governance standards. This combination of AI + Cloud-native engineering is helping us modernize faster — moving from reactive data management to proactive insight generation. 💭 Curious to know — Are you or your teams experimenting with GenAI for data engineering in regulated industries like healthcare or finance? How are you balancing automation, cloud scalability, and compliance? #Healthcare #DataEngineering #AWS #GenAI #ETL #DataPipelines #AIinHealthcare #BigData #AWSGlue #Redshift #S3 #DataOps #HL7 #FHIR #OpenToWork #JobSearch #CareerGrowth #HiringNow #DataEngineerJobs #TechCareers #JobSeeker #GenAI #AIinDataEngineering #MachineLearning #Automation #AIAssistedEngineering #TechInnovation #DataDriven #DigitalTransformation #CloudData #FutureOfData #TechLeadership #C2C #C2H

12

As a Biostatistician, you provide solutions—but nothing beats the power of asking the RIGHT QUESTIONS. Before any test, code, analysis, or interpretation… ✨ Start with clarity. Client: “Hey! Can you look at this data and tell me what you see?” Me: ▪️ What is your research question? ▪️ What is your hypothesis? ▪️ What’s the study design? ▪️ What are your specific aims? ▪️ Any potential confounders or stratifiers to consider? ▪️ What’s the primary outcome and key predictor(s)? 💡Arm yourself with the right questions today. That’s where real statistical consulting begins. #Biostatistics #ResearchDesign #StatsCoach #DataAnalysis #BiostatisticsConsulting #PhDLife #PublicHealthStats

11

3 Comments

We’re already seeing what AI can unlock for retention and engagement, we’re here to help build a more meaningful employee experience.

29