Google's Gemini AI Launch.

Source: Google

Machine Learning engineer Santiago tweets, "Google, this is embarrassing," criticizing one of Google's demo videos for the new AI model Gemini. The video, generating millions of impressions, depicts Gemini smoothly answering questions about images. However, controversy arises as it's revealed the video wasn't in real-time, sparking doubt about the model's features. Critics label it as 'fake' on social media.

Another point of contention is Gemini's performance on the MMLU benchmark. Google claims Gemini outperformed human experts, but skeptics like Brett Winton argue that certain prompting techniques were used, placing Gemini behind both humans and GPT-4.

Despite frustrations, accusing Google of 'lying' may be a stretch. The YouTube description acknowledges reduced latency and shortened outputs for brevity. Regarding MMLU, Google DeepMind's website notes the use of different prompting techniques.

CEO Aravind Sriniva of Perplexity AI offers a balanced view, acknowledging Gemini's cool features and comparing it favorably to GPT-4. While marketing was deemed overboard, Sriniva suggests that demos like the multimodal video will be feasible within a year.