AI, LLMs, and Industry Insights: An AMA Session with Chetanya Rastogi

In a recent AMA session hosted by TechKareer, we had the opportunity to learn from Chetanya Rastogi, an AI expert with extensive experience in both academia and industry.

Who is Chetanya Rastogi?

Chetanya's journey in AI has been diverse:

• Completed his bachelor's degree from IIT Roorkee.

• Worked at Samsung for 2-3 years with the Data Intelligence team.

• Attended Stanford for graduate studies, where he created and taught CS25.

• Gained experience at AI startups like:

  • Moveworks (focused on search).

  • Brevian (Making AI accessible to business users).

His experience in both research and real-world AI applications gives him a unique perspective on AI development, deployment, and challenges. Let’s deep dive into the AMA session.

Questions on AI Development and Model Training

Question: What is the first thing to consider when building an AI model?

Answer: The first consideration is whether a human can perform the task. This establishes a baseline and helps determine the feasibility of the AI project. Before starting on technical aspects, it’s essential to analyze the data to identify patterns and challenges. A strong end-to-end pipeline, including data gathering, model training, evaluation, and human feedback loops, is crucial for refining the system.

Question: How can AI help with improving image quality and identifying objects, particularly when dealing with blurry images from distant cameras?

Answer: For blurry, low-resolution images (e.g., farm security systems), de-noising and upscaling models like "stable diffusion" improve image clarity. Once the image is clearer, object detection models identify objects. For instance, a US-based farm with cameras 2.5 -- 3 km away captures images at 480x600 pixels, where subjects may occupy only 10–12 pixels. AI enhances identification and enables analytical tracking. Human feedback refines the system.

Question: What made Deepseek’s model training process so efficient?

Answer: Deepseek improved efficiency using:

• Multi-token prediction instead of single-token prediction.

• KV cache optimizations for faster access.

• A simple RL function that rewards correct answers, allowing them to generate synthetic training data.

Questions on Large Language Models (LLMs) and AI Agents

Question: What is the current market direction concerning Large Language Models (LLMs) and AI agents?

Answer: LLMs are becoming commoditized. Unless there’s a highly specific need, building a base model is inefficient. AI agents, defined as having a "central intelligent decision maker," automate tasks like debugging, invoice matching, and administration. However, agents are often unreliable due to non-standard human text and unpredictable user input. The focus should be on building domain-specific LLM applications.

Question: In your experience, which side have you been more exposed to, the creation of models or the infrastructure?

Answer: Experience has shifted over time. Before ChatGPT, the focus was on training and fine-tuning models. Now, the emphasis is on integrating and improving reliability in AI agent systems. While there is familiarity with AI infrastructure, expertise in kernel writing is limited.

Question: Can you provide some real-life examples of where agentic systems can be used, perhaps in daily life?

Answer: Agentic systems automate information-heavy tasks, such as:

• Summarizing long podcasts into short excerpts.

• Generating pull requests or commit messages in coding.

• Analyzing meeting transcripts or sales cycles.

Question: How do you build evaluations for something like a voice agent?

Answer: Evaluation methods include:

• Word error rate (WER) for speech accuracy.

• Conversation duration for interactive agents, measuring how long they sustain conversations.

Questions on AI in SaaS and Cloud Infrastructure

Question: What kind of AI-based software as a service (SaaS) products are worth developing?

Answer: Successful AI SaaS products focus on solving real problems in:

• Logistics

• CRM (Customer Relationship Management)

• Document processing

• Meeting summarization

Question: What type of SaaS product should we build?

Answer: The key is to understand industry problems before building solutions. AI SaaS opportunities exist in logistics, CRM, document processing, and meeting summarization. Without a deep understanding of the problem, delivering an effective solution is unlikely.

Question: When working with ERP systems, there are privacy concerns when using APIs. Additionally, self-hosting can be costly. Is there a way to ensure that the company hosting the LLM cannot access the data?

Answer: Complete data privacy is challenging since data must pass through APIs. Companies handling sensitive data can:

• Host models on-premises or in private cloud setups**.**

• Use smaller models on consumer-grade hardware**.**

Question: How should cloud providers evolve, considering the paradigm changes in AI with agent clouds and the new AI infrastructure service?

Answer: Cloud services may become more specialized. Innovations are making distributed computing more feasible, leveraging everyday computers for AI workloads.

Questions on Career and Skill Development in AI

Question: What specific skills should I focus on developing if I want to work in AI?

Answer: Skills depend on the role you choose:

• Model creation: Strong background in math and computer science.

• Model deployment at scale: Systems engineering expertise.

• AI product development: Strong software engineering skills.
  Regardless of specialization, software engineering and evaluation skills are crucial.

Question: What skills should a beginner focus on to build a stronger base in AI and tech?

Answer: It depends on the AI field of interest:

• Model development: Computer science and math.

• Running models at scale: Distributed systems knowledge.

• Building AI products: Strong software engineering skills.

• General AI knowledge: Understanding of input-output evaluation.

Question: How can a full-stack developer start with AI/ML?

Answer: It depends on the goal:

• For product building: Explore OpenAI, LangChain, or Llama Index cookbooks.

• For model fine-tuning/training: Follow experts like JLMR or Andrej Karpathy.

• For deep AI optimization: Study CUDA kernels and performance engineering.

Question: Is there a path where you don’t have to learn any math to work in AI?

Answer: Yes, if working at the application layer. Understanding how APIs work and selecting the best models is key. However, optimizing models, improving efficiency, or fine-tuning requires math and systems knowledge.

Questions on Higher Education and Career Pathways

Question: What advice do you have for someone applying to a US university for a Master’s degree?

Answer: Recommendation letters are crucial**.** Obtain letters from individuals familiar with your work and capable of providing specific examples. While GPA and statement of purpose matter, recommendation letters carry the most weight.

Question: Does having good projects matter in a Master’s application?

Answer: Yes, especially if the projects are impactful or recognized or you have presented your project to a wider audience.

Question: How do you crack big tech startups or high-growth startups?

Answer:

• Look for companies where you can learn from smart people.

• Small teams may skip coding tests, focusing instead on your potential impact.

• Larger startups are more likely to include coding tests in interviews.

Question: When does a company feel more bureaucratic versus more startup-like?

Answer: Bureaucracy can exist in small startups. The difference depends on personal preference for structured work environments versus dynamic, smaller teams focused on building new things.

Question on Privacy, Security, and Future of AI

Question: When using LLM APIs provided by other companies, how can I be sure that my data is protected?

Answer: Complete data privacy is impossible, but options include:

• Hosting models locally to avoid third-party exposure.

Using smaller models on personal hardware to reduce data leakage risks.

Conclusion

Chetanya’s insights highlight that AI isn’t just about building models ; it’s about solving real problems and creating reliable systems.

His advice for aspiring AI professionals:

• Stay motivated and work on projects that interest you.

• Build strong engineering skills.

• Stay involved in the AI and tech community to keep learning.

With the right mindset and skills, you’ll be well-prepared for the future of AI.

Want to dive deeper? Check out the TechKareer YouTube channel for the full session!