Multi-Modal AI Agents Training in Kraków, Poland

This course introduces participants to the design, development, and deployment of multi-modal AI agents that can interact and reason across text, images, and speech. It covers foundational concepts, tools, and frameworks for building real-world intelligent systems capable of natural communication, understanding, and decision-making across multiple data modalities.

Foundations of Multi-Modal AI

• Introduction to AI Agents and Multi-Modality

• Differences between single-modal vs. multi-modal AI

• Architectures for multi-modal learning (transformers, encoders, embeddings)

• Tools & frameworks: OpenAI GPT-4V, Hugging Face Transformers, LangChain

Text & Language Understanding in Multi-Modal Agents

• Advanced NLP for multi-modal agents

• Integrating structured and unstructured text data

• Prompt engineering strategies across modalities

Image & Visual Data in AI Agents

• Vision transformers (ViT), CLIP, and image captioning models

• Object detection, classification, and embeddings for agents

• Image-to-text and text-to-image (DALL·E, Stable Diffusion integration)

Speech & Audio in AI Agents

• Automatic Speech Recognition (ASR) with Whisper / Speech-to-Text APIs

• Text-to-Speech (TTS) synthesis for conversational agents

• Integrating speech with text and image inputs

Real-World Multi-Modal AI Agents

• Orchestrating text, image, and speech modalities in agents

• Workflow automation and decision-making with multi-modal inputs

Hands-on Exercises

Summary and Conclusion