Logo

SemiVoice

Recent #LLM news in the semiconductor industry

tweaktown
3 months ago

Apple fumbles AI again: new 'LLM Siri' update for iOS 19 delayed, now expected with iOS in 2026

➀ Apple's AI-powered Siri update, initially planned for iOS 19, has been delayed, highlighting Apple's struggles in AI development.

➁ The integration of advanced AI features is behind schedule, with a modernized Siri not expected until iOS 20 in 2027.

➂ Competitors have surpassed Apple, and internal challenges, including leadership and resource issues, hinder progress.

AIAI DevelopmentAlexaAppleChatGPTGrokInnovationLLMSiriiOS 19iOS 20technology
semiwiki
3 months ago

The Double-Edged Sword of AI Processors: Batch Sizes, Token Rates, and the Hardware Hurdles in Large Language Model Processing

➀ AI software modeling represents a significant shift from traditional programming, enabling systems to learn from data.

➁ The complexity of AI systems lies in their model parameters, which can number in the billions or trillions.

➂ GPUs have become essential for AI processing, but they face efficiency challenges, particularly during inference with large language models.

AIAI AcceleratorsASICComputational EfficiencyGPUHardwareLLMMemory Bandwidth
tweaktown
5 months ago

Researchers discover if 0.001% of AI training data misinformation the AI becomes corrupted

➀ Researchers found that even 0.001% misinformation in AI training data can compromise the entire system; ➁ The study injected AI-generated medical misinformation into a commonly used LLM training dataset, leading to a significant increase in harmful content; ➂ The researchers emphasized the need for better safeguards and security research in the development of medical LLMs.
AIAI CorruptionAI EthicsAI SecurityData MisinformationHealthcareLLM
tweaktown
8 months ago

Dell PowerEdge XE9712: NVIDIA GB200 NVL72-based AI GPU cluster for LLM training, inference

➀ Dell has launched the new PowerEdge XE9712 with NVIDIA GB200 NVL72 AI servers, offering 30x faster real-time LLM performance over the H100 AI GPU; ➁ The system features 72 x B200 AI GPUs connected with NVLink technology, providing lightning-fast connectivity; ➂ Dell highlights the liquid-cooled system for maximizing datacenter power utilization and rapid deployment of AI clusters.
AIData centerDellGPULLMNVIDIAPerformanceTraininginference
tweaktown
8 months ago

SK hynix starts mass production of 12-layer HBM3E memory: 36GB capacity per module @ 9.6Gbps

➀ SK hynix has begun mass production of the world's first 12-layer HBM3E memory with a capacity of up to 36GB and a bandwidth of 9.6Gbps; ➁ The new memory is designed for AI GPUs and is set to be supplied to NVIDIA within 12 months; ➂ SK hynix aims to maintain its leadership in AI memory with the introduction of this new technology.
AIAI GPUsBandwidthBlackwellH200HBM3EHopper H100LLMLlama 3 70BNVIDIASK hynixmemory
pcwatch
11 months ago

OpenAI Announces Cost-Effective LLM 'GPT-4o mini'

1. OpenAI introduces 'GPT-4o mini', a cost-effective language model priced at $0.15 per 1 million input tokens and $0.60 per 1 million output tokens, significantly cheaper than previous models and 60% cheaper than GPT-3.5 Turbo; 2. The model is designed for applications requiring low cost and low latency, such as chaining or parallelizing model calls, handling large amounts of context, and real-time text responses for customer interactions; 3. The current API supports text and vision, with plans to include video and audio input/output in the future, offering a 128K token context window and up to 16K output tokens per request, and knowledge up to October 2023. The model demonstrates improved cost-efficiency for non-English text processing with a shared, upgraded tokenizer with GPT-4o. It achieves high scores on various benchmarks, outperforming competitors like 'Gemini Flash' and 'Claude Haiku'.
Cost-EffectiveGPT-4o miniLLM
weixin
8 months ago

A New Chip Challenges GPU Performance

➀ A 30 billion parameter LLM is demonstrated with a prototype inference device equipped with 16 IBM AIU NorthPole processors, achieving a system throughput of 28,356 tokens/second and a latency below 1 ms/token; ➁ NorthPole offers 72.7 times better energy efficiency and lower latency compared to GPUs at the lowest GPU delay; ➂ NorthPole architecture is inspired by the brain, optimized for AI inference, and demonstrates superior performance in LLM推理.
GPULLMenergy efficiency
weixin
10 months ago

Year One of LLM Startup: Progress, Struggles, and Reflections

➀ The author reflects on the decision to start BosonAI, named after the particle in quantum physics, and the challenges in naming and branding. ➁ Details the financial rollercoaster, including a lead investor backing out at the last minute, and the subsequent successful completion of the funding round. ➂ Discusses the procurement of GPUs, highlighting the difficulties in obtaining H100s and the unexpected support from Nvidia's CEO. ➃ Shares the business achievements, including achieving a balanced budget in the first year and the potential for LLM applications in various industries. ➄ Explores the technical evolution of LLM understanding, from initial excitement to practical applications and the pursuit of specialized models. ➅ Outlines the vision for human companionship through AI, acknowledging the current limitations but expressing optimism for future developments. ➆ Emphasizes the importance of teamwork in entrepreneurship, contrasting the experience of working in a large corporation with the dynamics of a startup. ➇ Reflects on personal motivations for entrepreneurship, moving from a focus on fame and fortune to a deeper quest for creating meaningful value.
LLMStartuptechnology