Angela Gu

Apache Spark’s new "Real-time" Mode in Structured Streaming delivers millisecond-level latency for ultra-fast, continuous stream processing. It will be interesting to see how it will perform compared to Flink!

7

🚀 New Update: Dynamic Image Transformation for Amazon CloudFront We’re excited to roll out the latest version of Dynamic Image Transformation for Amazon CloudFront, making it easier than ever for teams to deliver fast, optimized, device-aware images through Amazon CloudFront's global CDN. This release introduces: ✨ Automatic image optimizations, tailoring image format, compression, and size to each user's device ✨ Support for delivering large, high-resolution images without compromising performance ✨ Support for external image origins, including popular CMS platforms ✨ An intuitive configuration experience to get up and running faster If you’re at AWS re:Invent this week, stop by the Edge & Networking area on the Expo floor to see a live demo and walkthrough of DIT. You can also explore other new CloudFront and Edge capabilities launching this week! https://lnkd.in/guc3pqhm

18

Building Large Language Models for the Public Sector on AWS Niki Sotiria Kokkalas Eliuth Triana Laura Verghote Wenhan Tan Excited to share our latest deep-dive on developing custom LLMs tailored for government and public sector organizations! 🏛️🤖 Off-the-shelf models often fall short of the regulatory, cultural, and operational requirements that define public sector missions. This comprehensive guide walks through the complete development lifecycle: ✅ Use case & requirements definition ✅ Evaluation framework establishment ✅ Model selection & training approach ✅ Data collection & preparation ✅ Infrastructure provisioning ✅ Production deployment & testing Key highlights: 🌍 National LLMs that preserve linguistic heritage & cultural context 🔒 Data sovereignty & compliance frameworks 💰 Cost analysis: API vs. self-hosting TCO 🛠️ Tools: NeMo Curator, SageMaker HyperPod, LM Harness Whether you're building language models for underrepresented languages or creating domain-specific solutions for healthcare, education, or legal services—this guide provides the scientific methodology and AWS infrastructure blueprint you need. Read the full article: https://lnkd.in/eKZ8FaKN Would love to hear your thoughts and experiences with custom LLM development! 💬

166

10 Comments

Over the past five months, I have completed three generative AI certifications from three of the largest AI companies: • AWS Certified AI Practitioner • Google Generative AI Leader Certification • NVIDIA Certified Associate in Generative AI LLMs I was asked how they differ and which one is “best”. The short answer is that they are designed for different use cases, even though they look similar on the surface. First, what they have in common All three cost roughly the same, around $100 per exam. All three are proctored, timed exams, either online with screen sharing or at a test centre. All three use a similar format of roughly 50 to 70 multiple-choice questions, with a pass mark of around 70 percent. Where they differ is what they are optimised to teach. AWS Certified AI Practitioner Of the three, this is comfortably the most hands-on and technical. AWS also provides the most thorough preparation by far. The learning pathways are free and combine theory, practice exams, interactive elements, and real hands-on building using services like Bedrock. You actively configure guardrails, choose knowledge bases, and build working GenAI applications. This is the best option if you want to understand the end-to-end process of building and deploying AI products in a real cloud environment. My preparation took around three weeks, a few hours a week, with significant practical work. Google Generative AI Leader Certification This is the most broad and accessible of the three. The training is open source, free, well designed, and suitable regardless of technical background. It focuses on understanding AI concepts, use cases, risks, and Google’s ecosystem, without requiring you to build anything yourself. This is ideal if you want to understand GenAI quickly, especially from a leadership or decision-making perspective. I completed the training and the exam on the same day. NVIDIA Certified Associate in Generative AI LLMs This is the most theoretical certification. It goes deep into algorithms, statistics, training data, attention-based models, and core machine learning concepts. It is very strong from an educational and conceptual standpoint. One caveat is that NVIDIA’s official training sits behind a paywall, with multiple paid modules. I chose to prepare using open-source material instead, which worked well. If you followed the full NVIDIA training path, this would likely be the longest and most expensive route. This is the best option if you want a deep theoretical understanding of how GenAI and ML models actually work, rather than how to deploy them. In summary If you want fast, high-level understanding and limited time, choose Google. If you want deep theoretical and educational grounding, choose NVIDIA. If you want hands-on, practical experience building real AI products, AWS is comfortably the strongest. Happy to compare notes with others who have taken a different path.

76

8 Comments

Thrilled to share the launch announcement for Google Cloud's Iceberg REST Catalog GA - the key to seamless engine choice! Data engineers can now use their preferred tool—BigQuery, Dataproc, Trino, or any Iceberg-compatible engine—on a single, highly performant, governed dataset. Get ready for a massive leap in productivity. 🧊🚀 #BigLake #Iceberg #GCP. Proud of the team that made this possible! Vinod RamachandranDeepak N.Garrett CastoGauri KalraGreg Smith Victor Agababov Ahmet Altay Samarth Shah Hannah Dunn & many others!!

17

OpenAI just released their first open weight models since GPT-2. You can try them out in Oracle Cloud Infrastructure (OCI) Data Science. https://lnkd.in/gsh--rdn

8

An agent has no consistent definition. You can think of an agent as an autonomous or semi-autonomous system that can take actions on behalf of the user in a given environment, state, and make decisions or take actions that can accomplish certain tasks along a given time-frame, either by calling tools, dynamically selecting the next action to take or use a deterministic "workflow" (systems where LLMs and tools are orchestrated through predefined code paths). Using an Agent framework that gives you the ability to built systems in both, a reliable and a dynamic way can accelerate your agent development journey using agent abstractions. This means building Agentic systems that can reliably call tools, store memory (episodic/semantic/procedural), have comprehensive logging and observability, human in the loop workflows and the ability to build various multi-agent patterns flexibly based on your use case. A successful Agentic system in production is usually a combination of both, dynamic and predictable/reliable multi-agent systems. Strands Agents SDK gives you exactly these capabilities by treating each “agent” as a combination of a foundation model plus a suite of tools. You define a prompt and register your tools (decorated functions) in code, then Strands handles reasoning→planning→tool-execution cycles, local testing, and cloud deployment (ECS, Fargate, Lambda, EC2), along with support for all other agent abstractions provided above. Excited to share a hedge-fund analyst multi agent system: This uses the newest Anthropic's Claude 4 Sonnet/Opus that powers the Lead Analyst Agent, routing incoming queries to specialized sub-agents for fundamental, technical, and market analyses. Each specialist is wrapped as a callable tool (using the “agents-as-tools” multi-agent pattern), so the orchestrator never has to implement domain logic itself and can handoff the task to an agent as a tool. For sensitive operations (insider lookups), we utilize a HITL approval step that halts execution until a human grants consent. We also use meta-tooling that enables the Lead Analyst to generate, load, and invoke new custom tools at runtime—whether it’s a portfolio beta calculator or a pricer—without redeployment. Strands also embeds observability (Langfuse) and OpenTelemetry tracing so you can trace reasoning events, tool invocations, errors, and end-to-end workflows in real time. View more information on the code implementation here: https://lnkd.in/gJmwVyGi Code implementation: https://lnkd.in/gzTtJvJq Thanks to 🏄‍♂️ Cagatay Cali for being a reviewer/collaborator on this! Feel free to try it out and reach out with any questions/ideas. #aws #agenticAI #strands #agents #generativeAI

60

6 Comments

Qodo and Google Cloud have announced a new collaboration aimed at improving code quality amidst the rapid acceleration of AI-driven software development. As part of this partnership, Qodo will integrate Google's Gemini models into its platform. #AI #SoftwareDevelopment #DevTools #OpenSource

1

Google is releasing fully-managed Model Context Protocol (MCP) support for Google and Google Cloud services. We are launching with support for key services to ground AI agents in the real world and reason over enterprise data: - Google Maps: Connects agents to trusted geospatial data. - BigQuery: Enables agents to interpret schemas and query enterprise data securely. - Google Compute Engine (GCE) & Google Kubernetes Engine (GKE): Empowers agents to autonomously manage infrastructure and container operations. This is a critical step in making agentic AI easier to build and more powerful for everyone. Read the full announcement to learn more about our commitment to building the best ecosystem for AI to thrive. #GoogleCloud #AI #AgenticAI #MCP #Developer #MachineLearning #API https://google.smh.re/5JJ9

1

One of my favorite parts of working in data is the community, and I'm thrilled to be presenting at the Bay Area Streaming Lakehouse Meetup next Wednesday (Feb 25) at Plug and Play, Sunnyvale, CA! I'll be talking about "Real-Time Analytics for End Users and Agents", exploring how real-time analytics is evolving to serve not just dashboards with pre-fixed queries, but also end users and AI agents with ad hoc queries directly. Joining an incredible lineup with Benjamin Wagner from Firebolt and Yingjun Wu from RisingWave. It's going to be a fun evening. If you're in the Bay Area, I'd love to catch up or meet you in person. Hope to see you there! 👋 🔗 https://luma.com/j56xvpd5 #RealTimeAnalytics #ApacheIceberg #StreamProcessing #Lakehouse #DataEngineering #BayAreaMeetup

32