Get live statistics and analysis of Vaibhav (VB) Srivastav's profile on X / Twitter

Let’s fucking goo!! DeepSeek R1 1.5B running FULLY LOCALLY in your browser at 60 tok/ sec powered by WebGPU🔥 Intelligence truly is too cheap to meter! ⚡️

WAIT A SECOND, DeepSeek just dropped Janus 7B (MIT Licensed) - multimodal LLM (capable of generating images too) 🔥

NEW: DeepSeek Janus Pro 1B (Generate Images, Chat with PDF) running in your browser, 100% local, powered by WebGPU 🔥 Zero server costs, brought to you by transformers.js - try it out!

"DeepSeek-R1-Distill-Qwen-1.5B outperforms GPT-4o and Claude-3.5-Sonnet on math benchmarks with 28.9% on AIME and 83.9% on MATH." 1.5B did WHAT?

🚨 Apple just released FastVLM on Hugging Face - 0.5, 1.5 and 7B real-time VLMs with WebGPU support 🤯 > 85x faster and 3.4x smaller than comparable sized VLMs > 7.9x faster TTFT for larger models > designed to output fewer output tokens and reduce encoding time for high resolution images Bonus: works in REALTIME directly in your browser powered by transformers.js and WebGPU 🔥 Try it out on the demo below 👇

Wow! Meta dropped an open NotebookLM recipe: NotebookLlama 🔥 It uses L3.2 1B/ 3B for pre-processing the PDF, L3.1 70B for Transcript creation, L3.1 8B for re-writes and Parler TTS for Text to Speech ⚡ Step 1: Pre-process PDF: Use Llama-3.2-1B-Instruct to pre-process the PDF and save it in a .txt file. Step 2: Transcript Writer: Use Llama-3.1-70B-Instruct to write a podcast transcript from the text Step 3: Dramatic Re-Writer: Use Llama-3.1-8B-Instruct model to make the transcript more dramatic Step 4: Text-To-Speech Workflow: Use parler-tts/parler-tts-mini-v1 and bark/suno to generate a conversational podcast There's still some rough edges, but it already sounds pretty fire - Link to notebook in the comments ✨

I did this. Fuck what anyone else says, just put the pedal to the metal and BUILD. Push spaghetti code. Nobody cares about OOPs. Doesn’t matter what anyone thinks. Just keep on doing. Document in public. Don’t listen to the haters. Release more than you refactor. Just keep pushing. Nobody cares, nobody notices, that’s the key, just fucking PUSH. That’s the way out, that’s what works, 100% times.

HOLY SHITT, Microsoft dropped an open-source Multimodal (supports Audio, Vision and Text) Phi 4 - MIT licensed! 🔥 > Beats Gemini 2.0 Flash, GPT4o, Whisper, SeamlessM4T v2 > Models on Hugging Face hub, integrated with/ Transformers! Phi-4-Multimodal: > Modalities: Integrates text, vision, and speech/audio > Architecture: Uses "Mixture of LoRAs" to add modality-specific adapters without fine-tuning the base model > Vision Modality: SigLIP-400M image encoder, 2-layer MLP projector, dynamic multi-crop strategy > Speech/Audio Modality: 3-layer convolution, 24 conformer blocks, 80ms token rate > Performance: Ranks first on OpenASR leaderboard, supports vision+language, vision+speech, and speech/audio tasks, outperforming larger models Phi-4-Mini: > Parameters: 3.8 billion > Architecture: 32 Transformer layers, 3,072 hidden state size, Group Query Attention (GQA) with 24 query heads and 8 key/value heads > Vocabulary: 200K tokens for multilingual support. Training Data: High-quality web and synthetic data, emphasizing math and coding > Performance: Outperforms similar-sized models and matches larger models (e.g., DeepSeek-Rl-Distill-Qwen-7B) on math and coding tasks Training Pipeline: > Language Training: Pre-training on 5 trillion tokens, post-training with function calling, summarization, and instruction-following data > Multimodal Training: Vision training (4 stages), speech/audio training (2 stages), and joint vision-speech training > Reasoning Training: Pre-trained on 60B CoT tokens, fine-tuned on 200K high-quality CoT samples, and DPO-trained on 300K preference samples Vision Benchmarks: > Outperforms Phi-3.5-Vision, Qwen2.5-VL, InternVL2.5, and matches Gemini and GPT-4o on tasks like chart understanding and OCR > Vision-Speech Benchmarks: Significantly outperforms InternOmni and Gemini-2.0-Flash Speech Benchmarks: > ASR: Achieves SOTA on CommonVoice, FLEURS, and Open ASR Leaderboard, surpassing WhisperV3 and SeamlessM4T > AST: Best performance on CoVoST2, comparable to GPT-4o on FLEURS > Speech Summarization: First open-source model with this capability, close to GPT-4o in quality Language Benchmarks: > Outperforms similar-sized models (Llama-3.2, Ministral) and matches larger models (Qwen2.5-7B) on math, reasoning, and coding tasks > Coding: Strong performance on HumanEval, MBPP, and BigCodeBench Reasoning Benchmarks: > Reasoning-enhanced Phi-4-Mini outperforms DeepSeek-Rl-Distill-Llama-8B and matches DeepSeek-Rl-Distill-Qwen-7B on AIME, MATH-500, and GPQA Diamond

Fuck yeah! Llama 3.2 3B running on your browser! 100% local, powered by WebGPU & MLC 🦙

holy fuck, these gigachads dropped 6 distilled models right from 1.5B to 70B 🔥

HOLY SHITT! Llasa TTS - Llama 3.2 fine-tune with ultra realistic audio 🔥 > supports voice cloning in English + Chinese > trained on 250K hours of audio > 1B, 3B model (8B soon) > emotional speech (happy, angry, sad, whisper) > open weights & works with transformers/ vllm

Microsoft open sourced MarkItDown - convert files to Markdown - perfect for using with LLMs! 🔥

350MB is all you need to get near SoTA TTS! 🤯

WAIT A SECOND, DeepSeek just dropped Janus 7B (MIT Licensed) - multimodal LLM (capable of generating images too) 🔥

Let’s fucking goo!! DeepSeek R1 1.5B running FULLY LOCALLY in your browser at 60 tok/ sec powered by WebGPU🔥 Intelligence truly is too cheap to meter! ⚡️

bro why is everyone making LLMs bounce balls in random fkn shapes!?

"DeepSeek-R1-Distill-Qwen-1.5B outperforms GPT-4o and Claude-3.5-Sonnet on math benchmarks with 28.9% on AIME and 83.9% on MATH." 1.5B did WHAT?

I did this. Fuck what anyone else says, just put the pedal to the metal and BUILD. Push spaghetti code. Nobody cares about OOPs. Doesn’t matter what anyone thinks. Just keep on doing. Document in public. Don’t listen to the haters. Release more than you refactor. Just keep pushing. Nobody cares, nobody notices, that’s the key, just fucking PUSH. That’s the way out, that’s what works, 100% times.

Wow! Meta dropped an open NotebookLM recipe: NotebookLlama 🔥 It uses L3.2 1B/ 3B for pre-processing the PDF, L3.1 70B for Transcript creation, L3.1 8B for re-writes and Parler TTS for Text to Speech ⚡ Step 1: Pre-process PDF: Use Llama-3.2-1B-Instruct to pre-process the PDF and save it in a .txt file. Step 2: Transcript Writer: Use Llama-3.1-70B-Instruct to write a podcast transcript from the text Step 3: Dramatic Re-Writer: Use Llama-3.1-8B-Instruct model to make the transcript more dramatic Step 4: Text-To-Speech Workflow: Use parler-tts/parler-tts-mini-v1 and bark/suno to generate a conversational podcast There's still some rough edges, but it already sounds pretty fire - Link to notebook in the comments ✨

Let's goo! F5-TTS 🔊 > Trained on 100K hours of data > Zero-shot voice cloning > Speed control (based on total duration) > Emotion based synthesis > Long-form synthesis > Supports code-switching > Best part: CC-BY license (commercially permissive)🔥 Diffusion based architecture: > Non-Autoregressive + Flow Matching with DiT > Uses ConvNeXt to refine text representation, alignment Synthesised: I was, like, talking to my friend, and she’s all, um, excited about her, uh, trip to Europe, and I’m just, like, so jealous, right? (Happy emotion) The TTS scene is on fire! 🐐

🚨 Apple just released FastVLM on Hugging Face - 0.5, 1.5 and 7B real-time VLMs with WebGPU support 🤯 > 85x faster and 3.4x smaller than comparable sized VLMs > 7.9x faster TTFT for larger models > designed to output fewer output tokens and reduce encoding time for high resolution images Bonus: works in REALTIME directly in your browser powered by transformers.js and WebGPU 🔥 Try it out on the demo below 👇

if money and time weren’t a problem - what would you actually be doing?

HOLY SHITT, Microsoft dropped an open-source Multimodal (supports Audio, Vision and Text) Phi 4 - MIT licensed! 🔥 > Beats Gemini 2.0 Flash, GPT4o, Whisper, SeamlessM4T v2 > Models on Hugging Face hub, integrated with/ Transformers! Phi-4-Multimodal: > Modalities: Integrates text, vision, and speech/audio > Architecture: Uses "Mixture of LoRAs" to add modality-specific adapters without fine-tuning the base model > Vision Modality: SigLIP-400M image encoder, 2-layer MLP projector, dynamic multi-crop strategy > Speech/Audio Modality: 3-layer convolution, 24 conformer blocks, 80ms token rate > Performance: Ranks first on OpenASR leaderboard, supports vision+language, vision+speech, and speech/audio tasks, outperforming larger models Phi-4-Mini: > Parameters: 3.8 billion > Architecture: 32 Transformer layers, 3,072 hidden state size, Group Query Attention (GQA) with 24 query heads and 8 key/value heads > Vocabulary: 200K tokens for multilingual support. Training Data: High-quality web and synthetic data, emphasizing math and coding > Performance: Outperforms similar-sized models and matches larger models (e.g., DeepSeek-Rl-Distill-Qwen-7B) on math and coding tasks Training Pipeline: > Language Training: Pre-training on 5 trillion tokens, post-training with function calling, summarization, and instruction-following data > Multimodal Training: Vision training (4 stages), speech/audio training (2 stages), and joint vision-speech training > Reasoning Training: Pre-trained on 60B CoT tokens, fine-tuned on 200K high-quality CoT samples, and DPO-trained on 300K preference samples Vision Benchmarks: > Outperforms Phi-3.5-Vision, Qwen2.5-VL, InternVL2.5, and matches Gemini and GPT-4o on tasks like chart understanding and OCR > Vision-Speech Benchmarks: Significantly outperforms InternOmni and Gemini-2.0-Flash Speech Benchmarks: > ASR: Achieves SOTA on CommonVoice, FLEURS, and Open ASR Leaderboard, surpassing WhisperV3 and SeamlessM4T > AST: Best performance on CoVoST2, comparable to GPT-4o on FLEURS > Speech Summarization: First open-source model with this capability, close to GPT-4o in quality Language Benchmarks: > Outperforms similar-sized models (Llama-3.2, Ministral) and matches larger models (Qwen2.5-7B) on math, reasoning, and coding tasks > Coding: Strong performance on HumanEval, MBPP, and BigCodeBench Reasoning Benchmarks: > Reasoning-enhanced Phi-4-Mini outperforms DeepSeek-Rl-Distill-Llama-8B and matches DeepSeek-Rl-Distill-Qwen-7B on AIME, MATH-500, and GPQA Diamond

Nvidia just open sourced Parakeet TDT 0.6B - the BEST Speech Recognition model on Open ASR Leaderboard 🔥 Can transcribe 60 minutes of audio in 1 second 🤯 600M parameters, with CC-BY-4.0 license (commercially permissive) Congrats Nvidia on the brilliant release and beating all major closed source giants too! 🤗

llms this, llms that! why aren't people releasing more audio stuff 😭 i want tts, asr, speech translation, voice cloning, text to audio, text to music, anything..